logfmt 0.0.9 → 0.1.0.beta.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rspec +1 -0
- data/CHANGELOG.md +19 -0
- data/Gemfile +3 -2
- data/README.md +151 -4
- data/Rakefile +33 -2
- data/bench.rb +6 -4
- data/lib/logfmt/logger.rb +74 -0
- data/lib/logfmt/parser.rb +100 -94
- data/lib/logfmt/version.rb +3 -1
- data/lib/logfmt.rb +12 -2
- data/logfmt.gemspec +37 -19
- metadata +35 -22
- data/.github/workflows/gempush.yml +0 -24
- data/.github/workflows/ruby.yml +0 -19
- data/.gitignore +0 -19
- data/spec/logfmt/parser_spec.rb +0 -159
- data/spec/spec_helper.rb +0 -16
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d8f2e9e3f3d271e1cad7120cb1fedc6a839050004a51ccd72a9e0ef019b242e6
|
4
|
+
data.tar.gz: 01035abc746fce52ac36c672bec2640a3c4d436f300b244d13b4e936b655030b
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: bb92737ee195acf40cfae4ae48b73895165568690984448e75d9bd4c3a3131f570721ebf0182778c959dc312182f6ebc9bcc0a6a26a877f09e24faa16f2377ff
|
7
|
+
data.tar.gz: 3df2ec4510aad12b18cbe6d1f2da2ffc7b8e3114bb3c66a4b74ae9d04965a5ba766a90db1b7510104295e0c133d36b2bff7190ac80bbbdfac85d770a4b1bb98d
|
data/.rspec
CHANGED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,19 @@
|
|
1
|
+
# Change Log
|
2
|
+
All notable changes to this project will be documented in this file.
|
3
|
+
|
4
|
+
The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/).
|
5
|
+
|
6
|
+
## \[Unreleased\]
|
7
|
+
|
8
|
+
## [0.1.0.beta.1] 2022-10-21
|
9
|
+
### Added
|
10
|
+
- Add `Logfmt::Logger` and `Logfmt::TaggedLogger`.
|
11
|
+
The later is distributed as its own gem, `logfmt-tagged_logger`, but lives in this repo.
|
12
|
+
|
13
|
+
## [0.0.10] 2022-04-30
|
14
|
+
### Changed
|
15
|
+
- Autoload the `Logfmt::Parser` when it's used, in preparation for the coming `Logfmt::Logger` and friends.
|
16
|
+
Alternatively you can eager-load it into memory: `require "logfmt/parser"`.
|
17
|
+
|
18
|
+
### Added
|
19
|
+
- This CHANGELOG file.
|
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -1,7 +1,154 @@
|
|
1
1
|
# Logfmt
|
2
2
|
|
3
|
-
|
3
|
+
Write and parse structured log lines in the [logfmt style][logfmt-blog].
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
|
5
|
+
## Installation
|
6
|
+
|
7
|
+
Add this line to your Gemfile:
|
8
|
+
|
9
|
+
```ruby
|
10
|
+
gem "logfmt"
|
11
|
+
```
|
12
|
+
|
13
|
+
And then install:
|
14
|
+
|
15
|
+
```bash
|
16
|
+
$ bundle
|
17
|
+
```
|
18
|
+
|
19
|
+
Or install it yourself:
|
20
|
+
|
21
|
+
```bash
|
22
|
+
$ gem install logfmt
|
23
|
+
```
|
24
|
+
|
25
|
+
### Versioning
|
26
|
+
|
27
|
+
This project adheres to [Semantic Versioning][semver].
|
28
|
+
|
29
|
+
## Usage
|
30
|
+
|
31
|
+
`Logfmt` is composed to two parts: writing structured log lines in the `logfmt` style, and parsing `logfmt`-style log lines.
|
32
|
+
|
33
|
+
While writing and parsing `logfmt` are related, we've found that it's common to only need to do one or there other in a single application.
|
34
|
+
To support that usage, `Logfmt` leverages Ruby's `autoload` to lazily load the `Logfmt::Parser` or `Logfmt::Logger` (and associated code) into memory.
|
35
|
+
In the general case that looks something like:
|
36
|
+
|
37
|
+
```ruby
|
38
|
+
require "logfmt"
|
39
|
+
|
40
|
+
Logfmt # This constant was already loaded, but neither Logfmt::Parser
|
41
|
+
# nor Logfmt::Logger constants are loaded. Yet.
|
42
|
+
|
43
|
+
Logfmt.parse("…")
|
44
|
+
# OR
|
45
|
+
Logfmt::Parser.parse("…")
|
46
|
+
|
47
|
+
# Either of the above will load the Logfmt::Parser constant.
|
48
|
+
# Similarly you can autoload the Logfmt::Logger via
|
49
|
+
|
50
|
+
Logfmt::Logger.new
|
51
|
+
```
|
52
|
+
|
53
|
+
If you want to eagerly load the logger or parser, you can do that by requiring them directly
|
54
|
+
|
55
|
+
### Parsing log lines
|
56
|
+
|
57
|
+
```ruby
|
58
|
+
require "logfmt/parser"
|
59
|
+
|
60
|
+
Logfmt::Parser.parse('foo=bar a=14 baz="hello kitty" cool%story=bro f %^asdf')
|
61
|
+
#=> {"foo"=>"bar", "a"=>14, "baz"=>"hello kitty", "cool%story"=>"bro", "f"=>true, "%^asdf"=>true}
|
62
|
+
```
|
63
|
+
|
64
|
+
### Writing log lines
|
65
|
+
|
66
|
+
The `Logfmt::Logger` is built on the stdlib `::Logger` and adheres to its API.
|
67
|
+
The primary difference is that `Logfmt::Logger` defaults to a `logfmt`-style formatter.
|
68
|
+
Specifically, a `Logfmt::Logger::KeyValueFormatter`, which results in log lines something like this:
|
69
|
+
|
70
|
+
```ruby
|
71
|
+
require "logfmt/logger"
|
72
|
+
|
73
|
+
logger = Logfmt::Logger.new($stdout)
|
74
|
+
|
75
|
+
logger.info(foo: "bar", a: 14, "baz" => "hello kitty", "cool%story" => "bro", f: true, "%^asdf" => true)
|
76
|
+
#=> time=2022-04-20T23:30:54.647403Z severity=INFO foo=bar a=14 baz="hello kitty" cool%story=bro f %^asdf
|
77
|
+
|
78
|
+
logger.debug("MADE IT HERE!")
|
79
|
+
#=> time=2022-04-20T23:33:44.912595Z severity=DEBUG msg="MADE IT HERE!"
|
80
|
+
```
|
81
|
+
|
82
|
+
#### Tagged log lines
|
83
|
+
|
84
|
+
The `logfmt-tagged_logger` gem adds support for Rails-style [tagged logging][tagged-logger].
|
85
|
+
This gem adds a `Logfmt::TaggedLogger` which is built on `ActiveSupport::TaggedLogger`, but emits the tags in logfmt-style, as key/value pairs.
|
86
|
+
For example
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
logger = Logfmt::TaggedLogger.new($stdout)
|
90
|
+
|
91
|
+
logger.tagged(source: "api") do
|
92
|
+
logger.info(foo: "bar")
|
93
|
+
end
|
94
|
+
|
95
|
+
#=> time=2022-04-20T23:33:44.912595Z severity=info source=api foo=bar"
|
96
|
+
```
|
97
|
+
|
98
|
+
You can also pass "bare" tags and they'll be collected and emitted under the `tags` key.
|
99
|
+
|
100
|
+
```ruby
|
101
|
+
logger = Logfmt::TaggedLogger.new($stdout)
|
102
|
+
|
103
|
+
logger.tagged("API", "1.2.3.4") do
|
104
|
+
logger.info(foo: "bar")
|
105
|
+
end
|
106
|
+
|
107
|
+
#=> time=2022-04-20T23:33:44.912595Z severity=info tags="[API] [1.2.3.4]" foo=bar"
|
108
|
+
```
|
109
|
+
|
110
|
+
It's likely more helpful and useful to use meaningful key/values for your tags, rather than bare tags.
|
111
|
+
|
112
|
+
#### Expected key/value transformations
|
113
|
+
|
114
|
+
When writing a log line with the `Logfmt::Logger::KeyValueFormatter` the keys and/or values will be transformed thusly:
|
115
|
+
|
116
|
+
* "Bare messages" (those with no key given when invoking the logger) will be wrapped in the `msg` key.
|
117
|
+
|
118
|
+
```ruby
|
119
|
+
logger.info("here")
|
120
|
+
#=> time=2022-04-20T23:33:49.912997Z severity=INFO msg=here
|
121
|
+
```
|
122
|
+
|
123
|
+
* Values, including bare messages, containing white space or control characters (spaces, tabs, newlines, emoji, etc…) will be wrapped in double quotes (`""`) and fully escaped.
|
124
|
+
|
125
|
+
```ruby
|
126
|
+
logger.info("👻 Boo!")
|
127
|
+
#=> time=2022-04-20T23:33:35.912595Z severity=INFO msg="\u{1F47B} Boo!"
|
128
|
+
|
129
|
+
logger.info(number: 42, with_quotes: %{These "are" 'quotes', OK?})
|
130
|
+
#=> time=2022-04-20T23:33:36.412183Z severity=INFO number=42 with_quotes="These \"are\" 'quotes', OK?"
|
131
|
+
```
|
132
|
+
|
133
|
+
* Floating point values are truncated to three digits.
|
134
|
+
|
135
|
+
* Time values are formatted as ISO8601 strings, with six digits sub-second precision.
|
136
|
+
|
137
|
+
* A value that is an Array is wrapped in square brackets, and then the above rules applied to each Array value.
|
138
|
+
This works well for arrays of simple values - like numbers, symbols, or simple strings.
|
139
|
+
But complex data structures will result in human mind-breaking escape sequences.
|
140
|
+
So don't do that.
|
141
|
+
Keep values simple.
|
142
|
+
|
143
|
+
```ruby
|
144
|
+
logger.info(an_array: [1, "two", :three])
|
145
|
+
#=> time=2022-04-20T23:33:36.412183Z severity=INFO an_array="[1, two, three]"
|
146
|
+
```
|
147
|
+
|
148
|
+
**NOTE**: it is **not** expected that log lines generated by `Logfmt` can be round-tripped by parsing the log line with `Logfmt`.
|
149
|
+
Specifically, this applies to Unicode and some control characters, as well as bare messages which will be wrapped in the `msg` key when writing.
|
150
|
+
Additionally, symbol keys will be parsed back into string keys.
|
151
|
+
|
152
|
+
[logfmt-blog]: https://brandur.org/logfmt "Structured log lines with key/value pairs"
|
153
|
+
[semver]: https://semver.org/spec/v2.0.0.html "Semantic Versioning 2.0.0"
|
154
|
+
[tagged-logger]: https://guides.rubyonrails.org/debugging_rails_applications.html#tagged-logging "Tagged Logging"
|
data/Rakefile
CHANGED
@@ -1,6 +1,37 @@
|
|
1
|
-
|
2
|
-
require 'rspec/core/rake_task'
|
1
|
+
# frozen_string_literal: true
|
3
2
|
|
3
|
+
require "bundler/gem_helper"
|
4
|
+
require "rspec/core/rake_task"
|
5
|
+
|
6
|
+
desc "Run all specs"
|
4
7
|
RSpec::Core::RakeTask.new(:spec)
|
5
8
|
|
9
|
+
namespace "logfmt" do
|
10
|
+
Bundler::GemHelper.install_tasks name: "logfmt"
|
11
|
+
end
|
12
|
+
|
13
|
+
# Inspired by how dotenv/dotenv-rails handles mulitple Gems in a single repo
|
14
|
+
class LogFmtTaggedLoggerGemHelper < Bundler::GemHelper
|
15
|
+
def guard_already_tagged
|
16
|
+
# noop
|
17
|
+
end
|
18
|
+
|
19
|
+
def tag_version
|
20
|
+
# noop
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
namespace "logfmt-tagged_logger" do
|
25
|
+
LogFmtTaggedLoggerGemHelper.install_tasks name: "logfmt-tagged_logger"
|
26
|
+
end
|
27
|
+
|
28
|
+
desc "Build logfmt and logfmt-tagged_logger into the pkg directory"
|
29
|
+
task build: ["logfmt:build", "logfmt-tagged_logger:build"]
|
30
|
+
|
31
|
+
desc "Build and install logfmt and logfmt-tagged_logger into system gems"
|
32
|
+
task install: ["logfmt:install", "logfmt-tagged_logger:install"]
|
33
|
+
|
34
|
+
desc "Create tag, build, and push logfmt and logfmt-tagged_logger to rubygems.org"
|
35
|
+
task release: ["logfmt:release", "logfmt-tagged_logger:release"]
|
36
|
+
|
6
37
|
task default: :spec
|
data/bench.rb
CHANGED
@@ -1,13 +1,15 @@
|
|
1
|
-
|
1
|
+
# frozen_string_literal: true
|
2
2
|
|
3
|
-
|
4
|
-
|
3
|
+
$LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
|
4
|
+
|
5
|
+
require "benchmark"
|
6
|
+
require "logfmt"
|
5
7
|
|
6
8
|
N = 1_000
|
7
9
|
line = 'foo=bar a=14 baz="hello kitty" ƒ=2h3s cool%story=bro f %^asdf'
|
8
10
|
|
9
11
|
Benchmark.bm(20) do |x|
|
10
|
-
x.report(
|
12
|
+
x.report("char-by-char") do
|
11
13
|
N.times do
|
12
14
|
Logfmt.parse(line)
|
13
15
|
end
|
@@ -0,0 +1,74 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative "../logfmt"
|
4
|
+
require "logger"
|
5
|
+
require "time"
|
6
|
+
|
7
|
+
module Logfmt
|
8
|
+
class Logger < ::Logger
|
9
|
+
def initialize(*args, **kwargs)
|
10
|
+
super
|
11
|
+
@formatter ||= KeyValueFormatter.new
|
12
|
+
end
|
13
|
+
|
14
|
+
class KeyValueFormatter < ::Logger::Formatter
|
15
|
+
def call(severity, timestamp, progname, msg)
|
16
|
+
%(time=#{format_datetime(timestamp)} severity=#{severity.ljust(5)}#{format_progname(progname)} #{format_message(msg)}\n)
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
def format_datetime(time)
|
22
|
+
time.utc.iso8601(6)
|
23
|
+
end
|
24
|
+
|
25
|
+
def format_message(msg)
|
26
|
+
return unless msg
|
27
|
+
|
28
|
+
if msg.respond_to?(:to_hash)
|
29
|
+
pairs = msg.to_hash.map { |k, v| format_pair(k, v) }
|
30
|
+
pairs.compact.join(" ")
|
31
|
+
else
|
32
|
+
format_pair("msg", msg)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
def format_pair(key, value)
|
37
|
+
return nil if value.nil?
|
38
|
+
|
39
|
+
# Return a bare key when the value is a `TrueClass`
|
40
|
+
return key if value == true
|
41
|
+
|
42
|
+
"#{key}=#{format_value(value)}"
|
43
|
+
end
|
44
|
+
|
45
|
+
def format_progname(progname)
|
46
|
+
return nil unless progname
|
47
|
+
|
48
|
+
# Format this pair like any other to ensure quoting, escaping, etc…,
|
49
|
+
# But we also need a leading space so we can interpolate the resulting
|
50
|
+
# key/value pair into our log line.
|
51
|
+
" #{format_pair(" progname", progname)}"
|
52
|
+
end
|
53
|
+
|
54
|
+
def format_value(value)
|
55
|
+
if value.is_a?(Float)
|
56
|
+
format("%.3f", value)
|
57
|
+
elsif value.is_a?(Time)
|
58
|
+
format_datetime(value)
|
59
|
+
elsif value.respond_to?(:to_ary)
|
60
|
+
format_value(
|
61
|
+
"[#{Array(value).map { |v| format_value(v) }.join(", ")}]"
|
62
|
+
)
|
63
|
+
else
|
64
|
+
# Interpolating due to a weird/subtle behaviour possible in #to_s.
|
65
|
+
# Namely, it's possible it doesn't actually return a String:
|
66
|
+
# https://github.com/ruby/spec/blob/3affe1e54fcd11918a242ad5d4a7ba895ee30c4c/language/string_spec.rb#L130-L141
|
67
|
+
value = "#{value}" # rubocop:disable Style/RedundantInterpolation
|
68
|
+
value = value.dump if value.match?(/[[:space:]]|[[:cntrl:]]/) # wrap in quotes and escape control characters
|
69
|
+
value
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
end
|
74
|
+
end
|
data/lib/logfmt/parser.rb
CHANGED
@@ -1,112 +1,118 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative "../logfmt"
|
4
|
+
|
1
5
|
module Logfmt
|
2
|
-
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
6
|
+
module Parser
|
7
|
+
GARBAGE = 0
|
8
|
+
KEY = 1
|
9
|
+
EQUAL = 2
|
10
|
+
IVALUE = 3
|
11
|
+
QVALUE = 4
|
7
12
|
|
8
|
-
|
9
|
-
|
10
|
-
|
13
|
+
def self.numeric?(s)
|
14
|
+
s.is_a?(Numeric) || s.to_s.match?(/\A[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?\Z/)
|
15
|
+
end
|
11
16
|
|
12
|
-
|
13
|
-
|
14
|
-
|
17
|
+
def self.integer?(s)
|
18
|
+
s.is_a?(Integer) || s.to_s.match?(/\A[-+]?[0-9]+\Z/)
|
19
|
+
end
|
15
20
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
end
|
31
|
-
if state == KEY
|
32
|
-
if c > ' ' && c != '"' && c != '='
|
33
|
-
state = KEY
|
34
|
-
key << c
|
35
|
-
elsif c == '='
|
36
|
-
output[key.strip] = true
|
37
|
-
state = EQUAL
|
38
|
-
else
|
39
|
-
output[key.strip] = true
|
40
|
-
state = GARBAGE
|
41
|
-
end
|
42
|
-
output[key.strip] = true if i >= line.length
|
43
|
-
next
|
44
|
-
end
|
45
|
-
if state == EQUAL
|
46
|
-
if c > ' ' && c != '"' && c != '='
|
47
|
-
value = c
|
48
|
-
state = IVALUE
|
49
|
-
elsif c == '"'
|
50
|
-
value = ''
|
51
|
-
escaped = false
|
52
|
-
state = QVALUE
|
53
|
-
else
|
54
|
-
state = GARBAGE
|
21
|
+
def self.parse(line)
|
22
|
+
output = {}
|
23
|
+
key, value = +"", +""
|
24
|
+
escaped = false
|
25
|
+
state = GARBAGE
|
26
|
+
i = 0
|
27
|
+
line.each_char do |c|
|
28
|
+
i += 1
|
29
|
+
if state == GARBAGE
|
30
|
+
if c > " " && c != '"' && c != "="
|
31
|
+
key = c
|
32
|
+
state = KEY
|
33
|
+
end
|
34
|
+
next
|
55
35
|
end
|
56
|
-
if
|
57
|
-
if
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
36
|
+
if state == KEY
|
37
|
+
if c > " " && c != '"' && c != "="
|
38
|
+
state = KEY
|
39
|
+
key << c
|
40
|
+
elsif c == "="
|
41
|
+
output[key.strip] = true
|
42
|
+
state = EQUAL
|
43
|
+
else
|
44
|
+
output[key.strip] = true
|
45
|
+
state = GARBAGE
|
62
46
|
end
|
63
|
-
output[key.strip] =
|
47
|
+
output[key.strip] = true if i >= line.length
|
48
|
+
next
|
64
49
|
end
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
value =
|
71
|
-
|
72
|
-
|
73
|
-
|
50
|
+
if state == EQUAL
|
51
|
+
if c > " " && c != '"' && c != "="
|
52
|
+
value = c
|
53
|
+
state = IVALUE
|
54
|
+
elsif c == '"'
|
55
|
+
value = +""
|
56
|
+
escaped = false
|
57
|
+
state = QVALUE
|
58
|
+
else
|
59
|
+
state = GARBAGE
|
60
|
+
end
|
61
|
+
if i >= line.length
|
62
|
+
if integer?(value)
|
63
|
+
value = value.to_i
|
64
|
+
elsif numeric?(value)
|
65
|
+
fvalue = value.to_f
|
66
|
+
value = fvalue if fvalue.finite?
|
67
|
+
end
|
68
|
+
output[key.strip] = value || true
|
74
69
|
end
|
75
|
-
|
76
|
-
state = GARBAGE
|
77
|
-
else
|
78
|
-
value << c
|
70
|
+
next
|
79
71
|
end
|
80
|
-
if
|
81
|
-
if
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
72
|
+
if state == IVALUE
|
73
|
+
if !(c > " " && c != '"')
|
74
|
+
if integer?(value)
|
75
|
+
value = value.to_i
|
76
|
+
elsif numeric?(value)
|
77
|
+
fvalue = value.to_f
|
78
|
+
value = fvalue if fvalue.finite?
|
79
|
+
end
|
80
|
+
output[key.strip] = value
|
81
|
+
state = GARBAGE
|
82
|
+
else
|
83
|
+
value << c
|
86
84
|
end
|
87
|
-
|
85
|
+
if i >= line.length
|
86
|
+
if integer?(value)
|
87
|
+
value = value.to_i
|
88
|
+
elsif numeric?(value)
|
89
|
+
fvalue = value.to_f
|
90
|
+
value = fvalue if fvalue.finite?
|
91
|
+
end
|
92
|
+
output[key.strip] = value
|
93
|
+
end
|
94
|
+
next
|
88
95
|
end
|
89
|
-
|
90
|
-
|
91
|
-
|
92
|
-
|
93
|
-
|
94
|
-
|
95
|
-
|
96
|
-
|
96
|
+
if state == QVALUE
|
97
|
+
if c == "\\"
|
98
|
+
escaped = true
|
99
|
+
value << "\\"
|
100
|
+
elsif c == '"'
|
101
|
+
if escaped
|
102
|
+
escaped = false
|
103
|
+
value.chop! << c
|
104
|
+
next
|
105
|
+
end
|
106
|
+
output[key.strip] = value
|
107
|
+
state = GARBAGE
|
108
|
+
else
|
97
109
|
escaped = false
|
98
|
-
value
|
99
|
-
next
|
110
|
+
value << c
|
100
111
|
end
|
101
|
-
|
102
|
-
state = GARBAGE
|
103
|
-
else
|
104
|
-
escaped = false
|
105
|
-
value << c
|
112
|
+
next
|
106
113
|
end
|
107
|
-
next
|
108
114
|
end
|
115
|
+
output
|
109
116
|
end
|
110
|
-
output
|
111
117
|
end
|
112
118
|
end
|
data/lib/logfmt/version.rb
CHANGED
data/lib/logfmt.rb
CHANGED
@@ -1,2 +1,12 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "logfmt/version"
|
4
|
+
|
5
|
+
module Logfmt
|
6
|
+
autoload(:Logger, "logfmt/logger")
|
7
|
+
autoload(:Parser, "logfmt/parser")
|
8
|
+
|
9
|
+
def self.parse(line)
|
10
|
+
const_get(:Parser).parse(line)
|
11
|
+
end
|
12
|
+
end
|
data/logfmt.gemspec
CHANGED
@@ -1,22 +1,40 @@
|
|
1
|
-
#
|
2
|
-
lib = File.expand_path('../lib', __FILE__)
|
3
|
-
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
-
require 'logfmt/version'
|
1
|
+
# frozen_string_literal: true
|
5
2
|
|
6
|
-
|
7
|
-
|
8
|
-
gem.version = Logfmt::VERSION
|
9
|
-
gem.authors = ['Timothée Peignier']
|
10
|
-
gem.email = ['timothee.peignier@tryphon.org']
|
11
|
-
gem.description = %q{Parse log lines in the logfmt style.}
|
12
|
-
gem.summary = %q{Parse logfmt messages.}
|
13
|
-
gem.homepage = 'https://github.com/cyberdelia/logfmt-ruby'
|
14
|
-
gem.license = 'MIT'
|
3
|
+
require_relative "lib/logfmt/version"
|
4
|
+
require "English"
|
15
5
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "logfmt"
|
8
|
+
spec.version = Logfmt::VERSION
|
9
|
+
spec.authors = ["Timothée Peignier"]
|
10
|
+
spec.email = ["timothee.peignier@tryphon.org"]
|
11
|
+
|
12
|
+
spec.summary = "Write and parse logfmt messages."
|
13
|
+
spec.description = "Write and parse log lines in the logfmt style."
|
14
|
+
spec.homepage = "https://github.com/cyberdelia/logfmt-ruby"
|
15
|
+
spec.license = "MIT"
|
16
|
+
spec.required_ruby_version = ">= 2.5.0"
|
17
|
+
|
18
|
+
spec.metadata = {
|
19
|
+
"bug_tracker_uri" => "#{spec.homepage}/issues",
|
20
|
+
"changelog_uri" => "#{spec.homepage}/blog/master/CHANGELOG.md",
|
21
|
+
"documentation_uri" => spec.homepage,
|
22
|
+
"source_code_uri" => spec.homepage
|
23
|
+
}
|
24
|
+
|
25
|
+
spec.files = Dir.chdir(File.expand_path(__dir__)) do
|
26
|
+
`git ls-files`.split($OUTPUT_RECORD_SEPARATOR)
|
27
|
+
.reject { |f|
|
28
|
+
(f == __FILE__) ||
|
29
|
+
f.match?(%r{\A(?:(?:bin|spec|features)/|\.(?:git|github))}) ||
|
30
|
+
f.match?(/tagged_logger/)
|
31
|
+
}
|
32
|
+
end
|
33
|
+
spec.bindir = "bin"
|
34
|
+
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
|
35
|
+
spec.require_paths = ["lib"]
|
36
|
+
|
37
|
+
spec.add_development_dependency "pry-byebug", "~> 3.9"
|
38
|
+
spec.add_development_dependency "rake", "~> 13.0"
|
39
|
+
spec.add_development_dependency "rspec", "~> 3.0"
|
22
40
|
end
|
metadata
CHANGED
@@ -1,68 +1,83 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: logfmt
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.1.0.beta.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Timothée Peignier
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2022-10-21 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
|
-
name:
|
14
|
+
name: pry-byebug
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
17
|
- - "~>"
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '3.
|
19
|
+
version: '3.9'
|
20
20
|
type: :development
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - "~>"
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '3.
|
26
|
+
version: '3.9'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: rake
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - "~>"
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '
|
33
|
+
version: '13.0'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '13.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rspec
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '3.0'
|
34
48
|
type: :development
|
35
49
|
prerelease: false
|
36
50
|
version_requirements: !ruby/object:Gem::Requirement
|
37
51
|
requirements:
|
38
52
|
- - "~>"
|
39
53
|
- !ruby/object:Gem::Version
|
40
|
-
version: '
|
41
|
-
description:
|
54
|
+
version: '3.0'
|
55
|
+
description: Write and parse log lines in the logfmt style.
|
42
56
|
email:
|
43
57
|
- timothee.peignier@tryphon.org
|
44
58
|
executables: []
|
45
59
|
extensions: []
|
46
60
|
extra_rdoc_files: []
|
47
61
|
files:
|
48
|
-
- ".github/workflows/gempush.yml"
|
49
|
-
- ".github/workflows/ruby.yml"
|
50
|
-
- ".gitignore"
|
51
62
|
- ".rspec"
|
63
|
+
- CHANGELOG.md
|
52
64
|
- Gemfile
|
53
65
|
- README.md
|
54
66
|
- Rakefile
|
55
67
|
- bench.rb
|
56
68
|
- lib/logfmt.rb
|
69
|
+
- lib/logfmt/logger.rb
|
57
70
|
- lib/logfmt/parser.rb
|
58
71
|
- lib/logfmt/version.rb
|
59
72
|
- logfmt.gemspec
|
60
|
-
- spec/logfmt/parser_spec.rb
|
61
|
-
- spec/spec_helper.rb
|
62
73
|
homepage: https://github.com/cyberdelia/logfmt-ruby
|
63
74
|
licenses:
|
64
75
|
- MIT
|
65
|
-
metadata:
|
76
|
+
metadata:
|
77
|
+
bug_tracker_uri: https://github.com/cyberdelia/logfmt-ruby/issues
|
78
|
+
changelog_uri: https://github.com/cyberdelia/logfmt-ruby/blog/master/CHANGELOG.md
|
79
|
+
documentation_uri: https://github.com/cyberdelia/logfmt-ruby
|
80
|
+
source_code_uri: https://github.com/cyberdelia/logfmt-ruby
|
66
81
|
post_install_message:
|
67
82
|
rdoc_options: []
|
68
83
|
require_paths:
|
@@ -71,17 +86,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
71
86
|
requirements:
|
72
87
|
- - ">="
|
73
88
|
- !ruby/object:Gem::Version
|
74
|
-
version:
|
89
|
+
version: 2.5.0
|
75
90
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
76
91
|
requirements:
|
77
|
-
- - "
|
92
|
+
- - ">"
|
78
93
|
- !ruby/object:Gem::Version
|
79
|
-
version:
|
94
|
+
version: 1.3.1
|
80
95
|
requirements: []
|
81
|
-
rubygems_version: 3.
|
96
|
+
rubygems_version: 3.3.7
|
82
97
|
signing_key:
|
83
98
|
specification_version: 4
|
84
|
-
summary:
|
85
|
-
test_files:
|
86
|
-
- spec/logfmt/parser_spec.rb
|
87
|
-
- spec/spec_helper.rb
|
99
|
+
summary: Write and parse logfmt messages.
|
100
|
+
test_files: []
|
@@ -1,24 +0,0 @@
|
|
1
|
-
name: Publish
|
2
|
-
on:
|
3
|
-
release:
|
4
|
-
types: [created]
|
5
|
-
jobs:
|
6
|
-
build:
|
7
|
-
name: Build + Publish
|
8
|
-
runs-on: ubuntu-latest
|
9
|
-
steps:
|
10
|
-
- uses: actions/checkout@master
|
11
|
-
- name: Set up Ruby 2.6
|
12
|
-
uses: actions/setup-ruby@v1
|
13
|
-
with:
|
14
|
-
version: 2.6.x
|
15
|
-
- name: Publish to RubyGems
|
16
|
-
run: |
|
17
|
-
mkdir -p $HOME/.gem
|
18
|
-
touch $HOME/.gem/credentials
|
19
|
-
chmod 0600 $HOME/.gem/credentials
|
20
|
-
printf -- "---\n:rubygems_api_key: ${GEM_HOST_API_KEY}\n" > $HOME/.gem/credentials
|
21
|
-
gem build *.gemspec
|
22
|
-
gem push *.gem
|
23
|
-
env:
|
24
|
-
GEM_HOST_API_KEY: ${{secrets.RUBYGEMS_AUTH_TOKEN}}
|
data/.github/workflows/ruby.yml
DELETED
@@ -1,19 +0,0 @@
|
|
1
|
-
name: CI
|
2
|
-
on: [push]
|
3
|
-
jobs:
|
4
|
-
build:
|
5
|
-
runs-on: ubuntu-latest
|
6
|
-
strategy:
|
7
|
-
matrix:
|
8
|
-
ruby: [ '2.4', '2.5', '2.6' ]
|
9
|
-
steps:
|
10
|
-
- uses: actions/checkout@v1
|
11
|
-
- name: Set up Ruby
|
12
|
-
uses: actions/setup-ruby@v1
|
13
|
-
with:
|
14
|
-
ruby-version: ${{ matrix.ruby }}
|
15
|
-
- name: Run tests
|
16
|
-
run: |
|
17
|
-
gem install bundler
|
18
|
-
bundle install --jobs 4 --retry 3
|
19
|
-
bundle exec rspec spec
|
data/.gitignore
DELETED
data/spec/logfmt/parser_spec.rb
DELETED
@@ -1,159 +0,0 @@
|
|
1
|
-
# encoding: utf-8
|
2
|
-
require 'spec_helper'
|
3
|
-
require 'logfmt/parser'
|
4
|
-
|
5
|
-
describe Logfmt do
|
6
|
-
it 'parse empty log line' do
|
7
|
-
data = Logfmt.parse('')
|
8
|
-
expect(data).to eq({})
|
9
|
-
end
|
10
|
-
|
11
|
-
it 'parse whitespace only log line' do
|
12
|
-
data = Logfmt.parse("\t")
|
13
|
-
expect(data).to eq({})
|
14
|
-
end
|
15
|
-
|
16
|
-
it 'parse key without value' do
|
17
|
-
data = Logfmt.parse('key')
|
18
|
-
expect(data).to eq('key' => true)
|
19
|
-
end
|
20
|
-
|
21
|
-
it 'parse key without value and whitespace' do
|
22
|
-
data = Logfmt.parse(' key ')
|
23
|
-
expect(data).to eq('key' => true)
|
24
|
-
end
|
25
|
-
|
26
|
-
it 'parse multiple single keys' do
|
27
|
-
data = Logfmt.parse('key1 key2')
|
28
|
-
expect(data).to eq('key1' => true, 'key2' => true)
|
29
|
-
end
|
30
|
-
|
31
|
-
it 'parse unquoted value' do
|
32
|
-
data = Logfmt.parse('key=value')
|
33
|
-
expect(data).to eq('key' => 'value')
|
34
|
-
end
|
35
|
-
|
36
|
-
it 'parse pairs' do
|
37
|
-
data = Logfmt.parse('key1=value1 key2=value2')
|
38
|
-
expect(data).to eq('key1' => 'value1', 'key2' => 'value2')
|
39
|
-
end
|
40
|
-
|
41
|
-
it 'parse mixed single/non-single pairs' do
|
42
|
-
data = Logfmt.parse('key1=value1 key2')
|
43
|
-
expect(data).to eq('key1' => 'value1', 'key2' => true)
|
44
|
-
end
|
45
|
-
|
46
|
-
it 'parse mixed pairs whatever the order' do
|
47
|
-
data = Logfmt.parse('key1 key2=value2')
|
48
|
-
expect(data).to eq('key1' => true, 'key2' => 'value2')
|
49
|
-
end
|
50
|
-
|
51
|
-
it 'parse quoted value' do
|
52
|
-
data = Logfmt.parse('key="quoted value"')
|
53
|
-
expect(data).to eq('key' => 'quoted value')
|
54
|
-
end
|
55
|
-
|
56
|
-
it 'parse escaped quote value ' do
|
57
|
-
data = Logfmt.parse('key="quoted \" value" r="esc\t"')
|
58
|
-
expect(data).to eq('key' => 'quoted " value', 'r' => 'esc\t')
|
59
|
-
end
|
60
|
-
|
61
|
-
it 'parse mixed pairs' do
|
62
|
-
data = Logfmt.parse('key1="quoted \" value" key2 key3=value3')
|
63
|
-
expect(data).to eq('key1' => 'quoted " value', 'key2' => true, 'key3' => 'value3')
|
64
|
-
end
|
65
|
-
|
66
|
-
it 'parse mixed characters pairs' do
|
67
|
-
data = Logfmt.parse('foo=bar a=14 baz="hello kitty" ƒ=2h3s cool%story=bro f %^asdf')
|
68
|
-
expect(data).to eq('foo' => 'bar', 'a' => 14, 'baz' => 'hello kitty',
|
69
|
-
'ƒ' => '2h3s', 'cool%story' => 'bro', 'f' => true, '%^asdf' => true)
|
70
|
-
end
|
71
|
-
|
72
|
-
it 'parse pair with empty quote' do
|
73
|
-
data = Logfmt.parse('key=""')
|
74
|
-
expect(data).to eq('key' => '')
|
75
|
-
end
|
76
|
-
|
77
|
-
# Currently, the value comes back as "true", which could mess up stats
|
78
|
-
# Really, only "true" should come back as "true"
|
79
|
-
# it 'parse 1 as integer type' do
|
80
|
-
# data = Logfmt.parse('key=1')
|
81
|
-
# expect(data['key'].class).to eq(Fixnum)
|
82
|
-
# end
|
83
|
-
|
84
|
-
it 'parse positive integer as integer type' do
|
85
|
-
data = Logfmt.parse('key=234')
|
86
|
-
expect(data['key']).to eq(234)
|
87
|
-
expect(data['key'].class).to eq(Fixnum)
|
88
|
-
end
|
89
|
-
|
90
|
-
it 'parse negative integer as integer type' do
|
91
|
-
data = Logfmt.parse('key=-3428')
|
92
|
-
expect(data['key']).to eq(-3428)
|
93
|
-
expect(data['key'].class).to eq(Fixnum)
|
94
|
-
end
|
95
|
-
|
96
|
-
it 'parse positive float as float type' do
|
97
|
-
data = Logfmt.parse('key=3.342')
|
98
|
-
expect(data['key']).to eq(3.342)
|
99
|
-
expect(data['key'].class).to eq(Float)
|
100
|
-
end
|
101
|
-
|
102
|
-
it 'parse negative float as float type' do
|
103
|
-
data = Logfmt.parse('key=-0.9934')
|
104
|
-
expect(data['key']).to eq(-0.9934)
|
105
|
-
expect(data['key'].class).to eq(Float)
|
106
|
-
end
|
107
|
-
|
108
|
-
it 'parse exponential float as float type' do
|
109
|
-
data = Logfmt.parse('key=2.342342342342344e+18')
|
110
|
-
expect(data['key']).to eq(2.342342342342344e+18)
|
111
|
-
expect(data['key'].class).to eq(Float)
|
112
|
-
end
|
113
|
-
|
114
|
-
it 'parse long digit string with embedded e as string' do
|
115
|
-
data = Logfmt.parse('key=2342342342342344e1818')
|
116
|
-
expect(data['key'].class).to eq(String)
|
117
|
-
end
|
118
|
-
|
119
|
-
it 'parse quoted integer as string type' do
|
120
|
-
data = Logfmt.parse('key="234"')
|
121
|
-
expect(data['key'].class).to eq(String)
|
122
|
-
end
|
123
|
-
|
124
|
-
it 'parse quoted float as string type' do
|
125
|
-
data = Logfmt.parse('key="3.14"')
|
126
|
-
expect(data['key'].class).to eq(String)
|
127
|
-
end
|
128
|
-
|
129
|
-
it 'parse IP address as string type' do
|
130
|
-
data = Logfmt.parse('key=10.10.10.1')
|
131
|
-
expect(data['key'].class).to eq(String)
|
132
|
-
end
|
133
|
-
|
134
|
-
it 'parse last as integer type' do
|
135
|
-
data = Logfmt.parse('key1=4 key2=9')
|
136
|
-
expect(data['key1']).to eq(4)
|
137
|
-
expect(data['key2']).to eq(9)
|
138
|
-
end
|
139
|
-
|
140
|
-
it 'parse string containing quotes' do
|
141
|
-
data = Logfmt.parse('key1="{\"msg\": \"hello\tworld\"}"')
|
142
|
-
expect(data['key1']).to eq('{"msg": "hello\tworld"}')
|
143
|
-
end
|
144
|
-
|
145
|
-
it 'parse value containing equal sign' do
|
146
|
-
query = 'position=44.80450799126121%2C33.58320759981871&uid=1'
|
147
|
-
data = Logfmt.parse("method=GET query=#{query} status=200")
|
148
|
-
expect(data).to eq(
|
149
|
-
'method' => 'GET',
|
150
|
-
'query' => query,
|
151
|
-
'status' => 200
|
152
|
-
)
|
153
|
-
end
|
154
|
-
|
155
|
-
it 'parses integers correctly' do
|
156
|
-
data = Logfmt.parse('key=111 ')
|
157
|
-
expect(data['key']).to eq(111)
|
158
|
-
end
|
159
|
-
end
|
data/spec/spec_helper.rb
DELETED
@@ -1,16 +0,0 @@
|
|
1
|
-
# This file was generated by the `rspec --init` command. Conventionally, all
|
2
|
-
# specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`.
|
3
|
-
# Require this file using `require "spec_helper"` to ensure that it is only
|
4
|
-
# loaded once.
|
5
|
-
#
|
6
|
-
# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
|
7
|
-
RSpec.configure do |config|
|
8
|
-
config.run_all_when_everything_filtered = true
|
9
|
-
config.filter_run :focus
|
10
|
-
|
11
|
-
# Run specs in random order to surface order dependencies. If you find an
|
12
|
-
# order dependency and want to debug it, you can fix the order by providing
|
13
|
-
# the seed, which is printed after each run.
|
14
|
-
# --seed 1234
|
15
|
-
config.order = 'random'
|
16
|
-
end
|