intake 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: febaaa600f3b7a5c8ed974a8e059a4d5ea12e61b0e5edf6fb0bf3c707f56027b
4
- data.tar.gz: 9bc82ce4ad485486a397e63c84c7a54445df78b2e3f285b1d0ec26d1cfddfdc8
3
+ metadata.gz: 0da496aeddc3a2af915242e66b0dde99077f9db6fad39b1aa608db88c4b3c7d7
4
+ data.tar.gz: 3e6f74f74938ca1ac38c4c6ffef7f6518a159ce208b26cd6fc442e1063e7d2b6
5
5
  SHA512:
6
- metadata.gz: bf360302e2ece8d18034a1b34587d3fa6c8a267f8d13ec64535b7de212e7307235b58b76a28c5f429fed832ee1ef4a1aa05c57391ba5cd74f98d207aa2c71033
7
- data.tar.gz: d140b351c1171e9a5bce2e5e4d01926900300f60a8e18ce0c0c591cadcf54ac6ced6fa22b584cf3d8ad4dff754dfc31399613ba39a1b2a8a750869026c430ab1
6
+ metadata.gz: 7b79c556355cd93395449943a59e1fda06af7081357fd94581a5ae8e192b10b3e5e113e8848b66c56cc9ab1b7e0242bbdbc1217c65c845dedf2e88f931e252c0
7
+ data.tar.gz: ed1959f3d7d111bd7acfb62412eec03c76767ef2cdf57ceab35ac5c0640dbc516cf5cf5e0f15c69367dab5a6f1ff7781264bcdd0f43f86c9a4aec95ba1f62929
data/README.md CHANGED
@@ -1,3 +1,5 @@
1
+ [![.github/workflows/rsspec.yml](https://github.com/the-vk/intake/actions/workflows/rspec.yml/badge.svg)](https://github.com/the-vk/intake/actions/workflows/rspec.yml)
2
+
1
3
  # intake
2
4
 
3
5
  ## Description
@@ -15,6 +17,198 @@ The library is designed with multiple base principles:
15
17
  gem install intake
16
18
  ```
17
19
 
20
+ ## Design
21
+
22
+ **intake** has two primary components:
23
+
24
+ * Logger
25
+ * Sink
26
+
27
+ **Intake::Logger** is a component that captures a logging event and forwards that to **Intake::EventDrain** which is a single point to collect log events.
28
+
29
+ A logger may optionally filter events by filter to quickly discard events with level below threshold.
30
+
31
+ **Intake::Sink** is a component that receives event logs and writes to a permanent storage.
32
+
33
+ A sink may filter events by level, logger name, or any other event attributes.
34
+
35
+ ## Examples
36
+
37
+ This example sets up logging to write messages to STDOUT.
38
+ **intake** writes messages in Ruby Logger format.
39
+ ```ruby
40
+ require 'intake`
41
+
42
+ log = Intake[:root]
43
+ log.level = :info
44
+ Intake.add_sink Intake::IOSink.new($stdout)
45
+
46
+ log.debug 'debug message'
47
+ log.debug { 'proc debug message' }
48
+ # Logger methods can take a block to generate log message
49
+ # Blocks allow to delay expensive message evaluation until and unless event is logged
50
+ log.debug do
51
+ print("you don't see me!")
52
+ 'expensive proc debug message'
53
+ end
54
+ log.info 'info message'
55
+ log.info { 'proc info message' }
56
+ log.warn 'warn message'
57
+ log.warn { 'proc warn message' }
58
+ log.error 'error message'
59
+ log.error { 'proc error message' }
60
+ log.fatal 'fatal message'
61
+ log.fatal { 'proc fatal message' }
62
+ ```
63
+
64
+ Output:
65
+
66
+ ```
67
+ I, [2022-10-09T15:57:58.377963 #12484] INFO -- : info message
68
+ I, [2022-10-09T15:57:58.378097 #12484] INFO -- : proc info message
69
+ W, [2022-10-09T15:57:58.378121 #12484] WARN -- : warn message
70
+ W, [2022-10-09T15:57:58.378133 #12484] WARN -- : proc warn message
71
+ E, [2022-10-09T15:57:58.378169 #12484] ERROR -- : error message
72
+ E, [2022-10-09T15:57:58.378212 #12484] ERROR -- : proc error message
73
+ F, [2022-10-09T15:57:58.378255 #12484] FATAL -- : fatal message
74
+ F, [2022-10-09T15:57:58.378297 #12484] FATAL -- : proc fatal message
75
+ ```
76
+
77
+ ### Sinks and filters
78
+
79
+ **intake** supports multuple target sinks with various filters.
80
+
81
+ ```ruby
82
+ log = Intake[:root]
83
+ log.level = :debug
84
+ io_sink = Intake::IOSink.new($stdout)
85
+ # filter is a proc-like object to make a decision on events
86
+ # event is accepted by sink if #call(event) returns true
87
+ io_sink.add_filter Intake::Filters::LevelFilter.new(:warn)
88
+ Intake.add_sink io_sink
89
+ file_sink = Intake::IOSink.new(File.new('/dev/null', 'a'))
90
+ file_sink.add_filter Intake::Filters::LevelFilter.new(:info)
91
+
92
+ log.debug 'debug message' # not logged
93
+ log.info 'info message' # logged to file
94
+ log.warn 'warn message' # logged to both stdout and file
95
+ log.error 'error message' # logged to both stdout and file
96
+ log.fatal 'fatal message' # logged to both stdout and file
97
+ ```
98
+
99
+ Output:
100
+
101
+ ```
102
+ W, [2022-10-09T16:08:48.752905 #14476] WARN -- : warn message
103
+ E, [2022-10-09T16:08:48.752983 #14476] ERROR -- : error message
104
+ F, [2022-10-09T16:08:48.753031 #14476] FATAL -- : fatal message
105
+ ```
106
+
107
+ ### MDC
108
+
109
+ Mapped diagnostic context (MDC) is a thread-local storage to attach extra information to log events, e.g. operation id or correlation id.
110
+
111
+ ```ruby
112
+ log = Intake[:root]
113
+ log.level = :info
114
+ sink = Intake::IOSink.new($stdout)
115
+ sink.formatter = ->(e) { "#{e.timestamp} [#{e[:correlation_id]}] - #{e.logger_name}: - #{e.message}\n" }
116
+ Intake.add_sink sink
117
+
118
+ log.info 'a message'
119
+
120
+ Intake::MDC[:correlation_id] = :abc
121
+
122
+ log.info 'message with MDC'
123
+
124
+ Intake::MDC.clear(:correlation_id)
125
+
126
+ log.info 'a message with no MDC'
127
+ ```
128
+
129
+ Output:
130
+
131
+ ```
132
+ 2022-10-09 16:13:23 -0700 [] - root: - a message
133
+ 2022-10-09 16:13:23 -0700 [abc] - root: - message with MDC
134
+ 2022-10-09 16:13:23 -0700 [] - root: - a message with no MDC
135
+ ```
136
+
137
+ ### Structured logging
138
+
139
+ **Intake::Logger** methods takes optional keyword argument **meta** with a Hash with extra details about log event.
140
+ Sink may write meta to output in structured format that allows to query logs.
141
+
142
+ ```ruby
143
+ log = Intake[:root]
144
+ log.level = :info
145
+ sink = Intake::IOSink.new($stdout)
146
+ sink.formatter = ->(e) { "#{e.timestamp} [#{e[:user_id]}] - #{e.logger_name}: - #{e.message}\n" }
147
+ Intake.add_sink sink
148
+
149
+ log.info 'a message', meta: { user_id: 'username' }
150
+ ```
151
+
152
+ Output:
153
+
154
+ ```
155
+ 2022-10-09 16:18:21 -0700 [username] - root: - a message
156
+ ```
157
+ ### Ruby Logger adapter
158
+
159
+ **intake** provides an adapter to Ruby Logger API. Adapter can be used as drop-in replacement of regular Ruby Logger.
160
+
161
+ ```ruby
162
+ require 'intake'
163
+
164
+ log = Intake[:root]
165
+ Intake.add_sink Intake::IOSink.new($stdout)
166
+ log.level = :debug
167
+ log = log.as_ruby_logger
168
+
169
+ log.add(Logger::Severity::FATAL, 'msg', 'sample')
170
+
171
+ log.debug 'debug'
172
+ log.info 'info'
173
+ log.warn 'warn'
174
+ log.error 'error'
175
+ log.fatal 'fatal'
176
+ log.unknown 'unknown'
177
+
178
+ log.warn { 'warn proc message' }
179
+
180
+ log = Intake[:root].as_ruby_logger(progname: 'sample')
181
+
182
+ log.debug 'debug'
183
+ log.info 'info'
184
+ log.warn 'warn'
185
+ log.error 'error'
186
+ log.fatal 'fatal'
187
+ log.unknown 'unknown'
188
+
189
+ log.warn { 'warn proc message' }
190
+ ```
191
+
192
+ Output:
193
+
194
+ ```
195
+ F, [2022-10-09T16:16:13.918872 #14918] FATAL -- sample: msg
196
+ D, [2022-10-09T16:16:13.918992 #14918] DEBUG -- : debug
197
+ I, [2022-10-09T16:16:13.919053 #14918] INFO -- : info
198
+ W, [2022-10-09T16:16:13.919114 #14918] WARN -- : warn
199
+ E, [2022-10-09T16:16:13.919143 #14918] ERROR -- : error
200
+ F, [2022-10-09T16:16:13.919191 #14918] FATAL -- : fatal
201
+ F, [2022-10-09T16:16:13.919219 #14918] FATAL -- : unknown
202
+ W, [2022-10-09T16:16:13.919267 #14918] WARN -- : warn proc message
203
+ D, [2022-10-09T16:16:13.919319 #14918] DEBUG -- sample: debug
204
+ I, [2022-10-09T16:16:13.919363 #14918] INFO -- sample: info
205
+ W, [2022-10-09T16:16:13.919417 #14918] WARN -- sample: warn
206
+ E, [2022-10-09T16:16:13.919442 #14918] ERROR -- sample: error
207
+ F, [2022-10-09T16:16:13.919491 #14918] FATAL -- sample: fatal
208
+ F, [2022-10-09T16:16:13.919542 #14918] FATAL -- sample: unknown
209
+ W, [2022-10-09T16:16:13.919594 #14918] WARN -- sample: warn proc message
210
+ ```
211
+
18
212
  ## License
19
213
 
20
214
  The MIT License. See the [LICENSE](/LICENSE) file for the full text.
@@ -0,0 +1,35 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Intake
4
+ # Base class that formats exception object to string
5
+ class ExceptionFormatter
6
+ attr_accessor :backtrace, :cause
7
+
8
+ def initialize
9
+ @backtrace = true
10
+ @cause = true
11
+ end
12
+
13
+ # Formats error to string
14
+ def call(err)
15
+ format(err, []).join("\n")
16
+ end
17
+
18
+ protected
19
+
20
+ def format(err, lines)
21
+ lines << format_title(err)
22
+ lines.concat(err.backtrace) if backtrace
23
+ format(err.cause, lines) if cause && !err.cause.nil?
24
+ lines
25
+ end
26
+
27
+ def format_title(err)
28
+ "Caused by: <#{err.class.name}> #{err.message}"
29
+ end
30
+
31
+ def format_backtrace(err, lines)
32
+ lines + err.backtrace
33
+ end
34
+ end
35
+ end
@@ -9,7 +9,7 @@ module Intake
9
9
 
10
10
  def call(event)
11
11
  # rubocop:disable Layout/LineLength
12
- "#{event.level.to_s[0]}, [#{event.timestamp.strftime(@timestamp_format)} ##{Process.pid}] #{event.level} -- #{event[:progname]}: #{event.message}\n"
12
+ "#{event.level.to_s[0]}, [#{event.timestamp.strftime(@timestamp_format)} ##{Process.pid}] #{event.level} -- #{event[:progname]}: #{event.message}"
13
13
  # rubocop:enable Layout/LineLength
14
14
  end
15
15
  end
@@ -1,5 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative 'exception_formatter'
3
4
  require_relative 'formatter'
4
5
  require_relative 'sink'
5
6
 
@@ -10,12 +11,14 @@ module Intake
10
11
  super()
11
12
  @io = io
12
13
  @formatter = ::Intake::Formatter.new
14
+ @exception_formatter = ::Intake::ExceptionFormatter.new
13
15
  end
14
16
 
15
- attr_writer :formatter
17
+ attr_accessor :formatter, :exception_formatter
16
18
 
17
19
  def drain(event)
18
- txt = @formatter.call(event)
20
+ error_message = "\n#{@exception_formatter.call(event[:error])}" unless event[:error].nil?
21
+ txt = "#{@formatter.call(event)}#{error_message}\n"
19
22
  @io.write(txt)
20
23
  end
21
24
  end
data/lib/intake/logger.rb CHANGED
@@ -11,31 +11,38 @@ require_relative 'repository'
11
11
  module Intake
12
12
  # Logger is a object that captures log event and forward that to event sinks.
13
13
  class Logger
14
+ DEFAULT_LEVEL = ::Intake::Level[:info]
14
15
  class << self
15
16
  def [](name)
16
- ::Intake::Repository.instance.get_or_add(name) do |logger_name|
17
- Logger.new(logger_name)
17
+ ::Intake::Repository.instance.get_or_add(name) do |logger_name, parent|
18
+ Logger.new(logger_name, parent: parent)
18
19
  end
19
20
  end
20
21
  end
21
22
 
22
23
  attr_reader :name
23
24
 
24
- def initialize(name)
25
+ def initialize(name, parent: nil)
25
26
  validate_name(name)
26
27
 
27
28
  @name = name
28
- @level = ::Intake::Level[:info]
29
+ @level = nil
30
+ @parent = parent
29
31
  end
30
32
 
31
33
  def level?(level)
32
- @level.val <= level.val
34
+ self.level.val <= level.val
35
+ end
36
+
37
+ def level
38
+ @level || @parent&.level || DEFAULT_LEVEL
33
39
  end
34
40
 
35
41
  def level=(level)
36
42
  @level = case level
37
43
  when String, Symbol then ::Intake::Level[level.to_sym]
38
44
  when ::Intake::Level then level
45
+ when nil then nil
39
46
  end
40
47
  end
41
48
 
@@ -19,17 +19,16 @@ module Intake
19
19
  @store.key? canonize_name(name)
20
20
  end
21
21
 
22
- def get_or_add(name)
23
- name = canonize_name(name)
24
- logger = @store[name]
22
+ def get_or_add(name, &block)
23
+ return nil if name.nil?
25
24
 
26
- if logger.nil?
27
- logger = @mutex.synchronize do
28
- @store[name] = yield(name) unless @store.key?(name)
29
- @store[name]
25
+ name = canonize_name(name)
26
+ unless @store.key? name
27
+ @mutex.synchronize do
28
+ create_logger_unsafe(name, &block) unless @store.key? name
30
29
  end
31
30
  end
32
- logger
31
+ @store[name]
33
32
  end
34
33
 
35
34
  private
@@ -44,8 +43,22 @@ module Intake
44
43
  end
45
44
  end
46
45
 
46
+ def parent_name(name)
47
+ separator_rindex = name.rindex('::')
48
+ separator_rindex.nil? ? 'root' : name[0, separator_rindex]
49
+ end
50
+
47
51
  def module_name(mod)
48
52
  mod.name
49
53
  end
54
+
55
+ def create_logger_unsafe(name, &block)
56
+ unless name == 'root'
57
+ parent_logger_name = parent_name(name)
58
+ parent_logger = @store[parent_logger_name] || create_logger_unsafe(parent_logger_name, &block)
59
+ end
60
+ @store[name] = block.call(name, parent_logger) unless @store.key?(name)
61
+ @store[name]
62
+ end
50
63
  end
51
64
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Intake
4
- VERSION = '0.1.0'
4
+ VERSION = '0.3.0'
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: intake
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrey Maraev
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-10-09 00:00:00.000000000 Z
11
+ date: 2022-10-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: concurrent-ruby
@@ -148,6 +148,7 @@ files:
148
148
  - lib/intake.rb
149
149
  - lib/intake/async_sink.rb
150
150
  - lib/intake/event_drain.rb
151
+ - lib/intake/exception_formatter.rb
151
152
  - lib/intake/filter.rb
152
153
  - lib/intake/filters/level_filter.rb
153
154
  - lib/intake/filters/logger_name_prefix_filter.rb