log2json-loggers 0.1.8 → 0.1.9

Sign up to get free protection for your applications and to get access to all the features.
Files changed (2) hide show
  1. data/lib/log2json/railslogger.rb +14 -5
  2. metadata +109 -60
@@ -26,14 +26,23 @@ module Log2Json
26
26
  # by a log2json log monitoring script.
27
27
  #
28
28
  def self.create_custom_rails_logger(config)
29
- path = config.paths["log"].first
30
- unless File.exist? File.dirname path
31
- FileUtils.mkdir_p File.dirname path
29
+ if config.respond_to?(:paths) # assume rails 3
30
+ path = config.paths["log"].first
31
+ unless File.exist? File.dirname path
32
+ FileUtils.mkdir_p File.dirname path
33
+ end
34
+ config.colorize_logging = false
35
+ else # assume rails 2
36
+ path = config.log_path
37
+ config.active_record.colorize_logging = false
32
38
  end
33
- config.colorize_logging = false
34
39
  logger = ::Logger.new(path)
35
40
  logger.formatter = ::Log2Json::log_formatter
36
- ActiveSupport::TaggedLogging.new(logger)
41
+ if defined?(ActiveSupport::TaggedLogging)
42
+ ActiveSupport::TaggedLogging.new(logger)
43
+ else
44
+ logger
45
+ end
37
46
  end
38
47
 
39
48
  # Simiar to the custom rails logger, but for unicorn.
metadata CHANGED
@@ -1,85 +1,134 @@
1
- --- !ruby/object:Gem::Specification
1
+ --- !ruby/object:Gem::Specification
2
2
  name: log2json-loggers
3
- version: !ruby/object:Gem::Version
4
- version: 0.1.8
3
+ version: !ruby/object:Gem::Version
4
+ hash: 9
5
5
  prerelease:
6
+ segments:
7
+ - 0
8
+ - 1
9
+ - 9
10
+ version: 0.1.9
6
11
  platform: ruby
7
- authors:
12
+ authors:
8
13
  - Jack Kuan
9
14
  autorequire:
10
15
  bindir: bin
11
16
  cert_chain: []
12
- date: 2013-10-02 00:00:00.000000000 Z
17
+
18
+ date: 2013-10-04 00:00:00 Z
13
19
  dependencies: []
14
- description: ! "Log2json lets you read, filter and send logs as JSON objects via Unix
15
- pipes.\nIt is inspired by Logstash, and is meant to be compatible with it at the
16
- JSON\nevent/record level so that it can easily work with Kibana. \n\nReading logs
17
- is done via a shell script(eg, `tail`) running in its own process.\nYou then configure(see
18
- the `syslog2json` or the `nginxlog2json` script for\nexamples) and run your filters
19
- in Ruby using the `Log2Json` module and its\ncontained helper classes.\n\n`Log2Json`
20
- reads from stdin the logs(one log record per line), parses the log\nlines into JSON
21
- records, and then serializes and writes the records to stdout,\nwhich then can be
22
- piped to another process for processing or sending it to\nsomewhere else.\n\nCurrently,
23
- Log2json ships with a `tail-log` script that can be run as the input\nprocess. It's
24
- the same as using the Linux `tail` utility with the `-v -F`\noptions except that
25
- it also tracks the positions(as the numbers of lines read\nfrom the beginning of
26
- the files) in a few files in the file system so that if the\ninput process is interrupted,
27
- it can continue reading from where it left off\nnext time if the files had been
28
- followed. This feature is similar to the sincedb\nfeature in Logstash's file input.\n\nNote:
29
- If you don't need the tracking feature(ie, you are fine with always\ntailling from
30
- the end of file with `-v -F -n0`), then you can just use the `tail`\nutility that
31
- comes with your Linux distribution.(Or more specifically, the\n`tail` from the GNU
32
- coreutils). Other versions of the `tail` utility may also\nwork, but are not tested.
33
- The input protocol expected by Log2json is very\nsimple and documented in the source
34
- code.\n\n** The `tail-log` script uses a patched version of `tail` from the GNU
35
- coreutils\n package. A binary of the `tail` utility compiled for Ubuntu 12.04
36
- LTS is\n included with the Log2json gem. If the binary doesn't work for your\n
37
- \ distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it\n
38
- \ can be found in the src/ directory of the installed gem), and then replace\n
39
- \ the bin/tail binary in the directory of the installed gem with your version\n
40
- \ of the binary. ** \n\nP.S. If you know of a way to configure and compile ONLY
41
- the tail program in\n coreutils, please let me know! The reason I'm not building
42
- tail post gem\n installation is that it takes too long to configure && make
43
- because that\n actually builds every utilties in coreutils.\n\n\nFor shipping
44
- logs to Redis, there's the `lines2redis` script that can be used as\nthe output
45
- process in the pipe. For shipping logs from Redis to ElasticSearch,\nLog2json provides
46
- a `redis2es` script.\n\nFinally here's an example of Log2json in action:\n\nFrom
47
- a client machine:\n\n tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log
48
- | syslog2json |\n queue=jsonlogs \\\n flush_size=20 \\\n flush_interval=30
49
- \\\n lines2redis host.to.redis.server 6379 0 # use redis DB 0\n\n\nOn the
50
- Redis server:\n\n redis_queue=jsonlogs redis2es host.to.es.server\n\n\n\nResources
51
- that help writing log2json filters:\n - look at log2json.rb source and example
52
- filters\n - http://grokdebug.herokuapp.com/\n - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime\n\n"
20
+
21
+ description: |+
22
+ Log2json lets you read, filter and send logs as JSON objects via Unix pipes.
23
+ It is inspired by Logstash, and is meant to be compatible with it at the JSON
24
+ event/record level so that it can easily work with Kibana.
25
+
26
+ Reading logs is done via a shell script(eg, `tail`) running in its own process.
27
+ You then configure(see the `syslog2json` or the `nginxlog2json` script for
28
+ examples) and run your filters in Ruby using the `Log2Json` module and its
29
+ contained helper classes.
30
+
31
+ `Log2Json` reads from stdin the logs(one log record per line), parses the log
32
+ lines into JSON records, and then serializes and writes the records to stdout,
33
+ which then can be piped to another process for processing or sending it to
34
+ somewhere else.
35
+
36
+ Currently, Log2json ships with a `tail-log` script that can be run as the input
37
+ process. It's the same as using the Linux `tail` utility with the `-v -F`
38
+ options except that it also tracks the positions(as the numbers of lines read
39
+ from the beginning of the files) in a few files in the file system so that if the
40
+ input process is interrupted, it can continue reading from where it left off
41
+ next time if the files had been followed. This feature is similar to the sincedb
42
+ feature in Logstash's file input.
43
+
44
+ Note: If you don't need the tracking feature(ie, you are fine with always
45
+ tailling from the end of file with `-v -F -n0`), then you can just use the `tail`
46
+ utility that comes with your Linux distribution.(Or more specifically, the
47
+ `tail` from the GNU coreutils). Other versions of the `tail` utility may also
48
+ work, but are not tested. The input protocol expected by Log2json is very
49
+ simple and documented in the source code.
50
+
51
+ ** The `tail-log` script uses a patched version of `tail` from the GNU coreutils
52
+ package. A binary of the `tail` utility compiled for Ubuntu 12.04 LTS is
53
+ included with the Log2json gem. If the binary doesn't work for your
54
+ distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it
55
+ can be found in the src/ directory of the installed gem), and then replace
56
+ the bin/tail binary in the directory of the installed gem with your version
57
+ of the binary. **
58
+
59
+ P.S. If you know of a way to configure and compile ONLY the tail program in
60
+ coreutils, please let me know! The reason I'm not building tail post gem
61
+ installation is that it takes too long to configure && make because that
62
+ actually builds every utilties in coreutils.
63
+
64
+
65
+ For shipping logs to Redis, there's the `lines2redis` script that can be used as
66
+ the output process in the pipe. For shipping logs from Redis to ElasticSearch,
67
+ Log2json provides a `redis2es` script.
68
+
69
+ Finally here's an example of Log2json in action:
70
+
71
+ From a client machine:
72
+
73
+ tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log | syslog2json |
74
+ queue=jsonlogs \
75
+ flush_size=20 \
76
+ flush_interval=30 \
77
+ lines2redis host.to.redis.server 6379 0 # use redis DB 0
78
+
79
+
80
+ On the Redis server:
81
+
82
+ redis_queue=jsonlogs redis2es host.to.es.server
83
+
84
+
85
+
86
+ Resources that help writing log2json filters:
87
+ - look at log2json.rb source and example filters
88
+ - http://grokdebug.herokuapp.com/
89
+ - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime
90
+
53
91
  email: jack.kuan@thescore.com
54
92
  executables: []
93
+
55
94
  extensions: []
95
+
56
96
  extra_rdoc_files: []
57
- files:
97
+
98
+ files:
58
99
  - lib/log2json/railslogger.rb
59
100
  homepage:
60
101
  licenses: []
102
+
61
103
  post_install_message:
62
104
  rdoc_options: []
63
- require_paths:
105
+
106
+ require_paths:
64
107
  - lib
65
- required_ruby_version: !ruby/object:Gem::Requirement
108
+ required_ruby_version: !ruby/object:Gem::Requirement
66
109
  none: false
67
- requirements:
68
- - - ! '>='
69
- - !ruby/object:Gem::Version
70
- version: '0'
71
- required_rubygems_version: !ruby/object:Gem::Requirement
110
+ requirements:
111
+ - - ">="
112
+ - !ruby/object:Gem::Version
113
+ hash: 3
114
+ segments:
115
+ - 0
116
+ version: "0"
117
+ required_rubygems_version: !ruby/object:Gem::Requirement
72
118
  none: false
73
- requirements:
74
- - - ! '>='
75
- - !ruby/object:Gem::Version
76
- version: '0'
119
+ requirements:
120
+ - - ">="
121
+ - !ruby/object:Gem::Version
122
+ hash: 3
123
+ segments:
124
+ - 0
125
+ version: "0"
77
126
  requirements: []
127
+
78
128
  rubyforge_project:
79
- rubygems_version: 1.8.23
129
+ rubygems_version: 1.8.15
80
130
  signing_key:
81
131
  specification_version: 3
82
- summary: Custom loggers for Rails and Unicorn that use log2json's single-line log
83
- format.
132
+ summary: Custom loggers for Rails and Unicorn that use log2json's single-line log format.
84
133
  test_files: []
85
- has_rdoc:
134
+