remote_syslog 1.4.1 → 1.4.2

data/README.md

@@ -27,8 +27,8 @@ Install the gem, which includes a binary called "remote_syslog":
     $ [sudo] gem install remote_syslog
 
 Optionally, create a log_files.yml with the log file paths to read and the 
-host/port to log to (see examples/log_files.yml.example). These can also be 
-specified as arguments to the remote_syslog daemon. More below.
+host/port to log to (see examples/[log_files.yml.example][sample config]). These can also be 
+specified as command-line arguments (below).
 
 
 ## Usage
@@ -38,7 +38,7 @@ specified as arguments to the remote_syslog daemon. More below.
 
     Example: remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log
 
-    Options:
+    Options (default):
         -c, --configfile PATH            Path to config (/etc/log_files.yml)
         -d, --dest-host HOSTNAME         Destination syslog hostname or IP (logs.papertrailapp.com)
         -p, --dest-port PORT             Destination syslog port (514)
@@ -56,18 +56,23 @@ specified as arguments to the remote_syslog daemon. More below.
 
 ## Example
 
-Daemonize, collecting from files mentioned in `./config/logs.yml` as well as
-`/var/log/mysqld.log`:
+Typical:
+
+    $ remote_syslog
+
+Daemonize and collect messages from files listed in `./config/logs.yml` as 
+well as the file `/var/log/mysqld.log`. Send to port `logs.papertrailapp.com:12345`:
 
     $ remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log
 
 Stay attached to the terminal, look for and use `/etc/log_files.yml` if it
-exists, write PID to `/tmp/remote_syslog.pid`, and send with facility local0:
+exists, write PID to `/tmp/remote_syslog.pid`, and send with facility local0
+to `a.server.com:514`:
 
-    $ remote_syslog -d a.server.com -f local0 -P /tmp /var/log/mysqld.log
+    $ remote_syslog -D -d a.server.com -f local0 -P /tmp /var/log/mysqld.log
 
 remote_syslog will daemonize by default. A sample init file is in the gem as 
-remote_syslog.init.d. You may be able to:
+[remote_syslog.init.d]. You may be able to:
 
     $ cp examples/remote_syslog.init.d /etc/init.d/remote_syslog
 
@@ -78,12 +83,11 @@ pass the `--tls` option when running `remote_syslog`:
 
     $ remote_syslog --tls -p 1234 /var/log/mysqld.log
 
-
 ## Configuration
 
 By default, the gem looks for a configuration in /etc/log_files.yml.
 
-The gem comes with a sample config.  Optionally:
+The gem comes with a [sample config].  Optionally:
 
     $ cp examples/log_files.yml.example /etc/log_files.yml
     
@@ -99,55 +103,75 @@ Only 1 destination server is supported; the command-line argument wins.
       host: logs.papertrailapp.com
       port: 12345
 
+remote_syslog sends the name of the file without a path ("mysqld.log") as 
+the syslog tag (program name). RFCs 3164 and 5424 limit the tag to 32 
+characters. Longer filenames are truncated to 32 characters.
+
+## Advanced Configuration (Optional)
+
+Here's an [advanced config] which uses all options.
+
+### Override hostname
+
+Provide `--hostname somehostname` or use the `hostname` configuration option:
+
+    hostname: somehostname
+
+### Multiple instances
+
+Run multiple instances to support more than one message-specific file format 
+or to specify unique syslog hostnames.
+
+To do that, provide an alternate PID filename as a command-line option
+to the additional instance(s). For example:
 
-### Optional: Parse fields from messages written by syslogd
+    --pid-file remote_syslog_2.pid
+
+### Parse fields from log messages
 
-This is not needed for most configurations.
+Rarely needed. Usually only used when remote_syslog is watching files 
+generated by syslogd (rather than by apps), like ``/var/log/messages``.
 
-In cases where logs from multiple programs are in the same file (for example, 
-``/var/log/messages``), the log line may include text that is not part of the 
-log message, like a timestamp, hostname, or program name. remote_syslog can 
-parse the program, hostname, and/or message text so that the message has 
-accurate metadata.
+remote_syslog can parse the program and hostname from the log line. When one 
+file contains logs from multiple programs (like with syslog), the log line 
+may include text that is not part of the log message, like a timestamp, 
+hostname, or program name. remote_syslog will extract those and use them in 
+the corresponding syslog packet fields.
 
-To do that, add an optional top-level configuration option `parse_fields` 
-with the name of a predefined regex (by remote_syslog) or a regex string. To 
-use the predefined regex for standard syslog messages, provide:
+To do that, use the config file option `parse_fields` with the name of a 
+format supported by remote_syslog, or your own regex. Included format names
+are `syslog` and `rfc3339`. For example:
 
     parse_fields: syslog
 
-The `syslog` regex is `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)`. It parses this:
+The included `syslog` format uses the regex `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)` 
+to parse standard syslog lines like this:
 
     Jul 18 08:25:08 hostname programname[1234]: The log message
 
-Or provide `parse_fields: rfc3339` to parse high-precision RFC 3339
-timestamps like:
-    2011-07-16T08:25:08.651413-07:00 hostname programname[1234]: The log message
-    
-Or provide your own regex that includes these 4 backreferences, in order: 
-timestamp, system name, program name, message. Match and return empty 
-strings for any empty positions where the log value should be ignored.
-For example, in the log:
+The included `rfc3339` format uses the regex `(\S+) (\S+) ([^: ]+):? (.*)` to 
+parse syslog lines with high-precision RFC 3339 timestamps, like this:
 
-    something-meaningless The log message
+    2011-07-16T08:25:08.651413-07:00 hostname programname[1234]: The log message
 
-You could ignore the first word, returning 3 empty values then the log 
-message with:
+To parse a format other than those, provide your own regex. It should include 
+4 backreferences to parse, in order: timestamp, system name, program name, 
+message.
 
-    parse_fields: "something-meaningless ()()()(.*)"
+Match and return empty strings for any empty positions where the log line 
+doesn't provide a value. For example, given the log message:
 
-Per-file parsing is not supported. Run multiple instances.
+    something-meaningless The log message
 
+One could use a regex to ignore "something-meaningless" (and not to extract
+a program or hostname). To ignore that prefix and return 3 empty values 
+then the log message, use parse_fields with this regex:
 
-### Optional: Run multiple instances
+    parse_fields: "something-meaningless ()()()(.*)"
 
-Run multiple instances to support more than one message-specific file format 
-(concurrently) or to specify distinct syslog hostnames. To do so, provide
-an alternative PID filename as a command-line option to additional 
-instance(s), such as:
+Per-file regexes are not supported. Run multiple instances with different 
+config files.
 
-    --pid-file remote_syslog_2.pid
-    
 
 ## Reporting bugs
 
@@ -166,6 +190,9 @@ Once you've made your great commits:
 5. Create a Pull Request or an [Issue][is] with a link to your branch
 6. That's it!
 
+[sample config]: https://github.com/papertrail/remote_syslog/blob/master/examples/log_files.yml.example
+[remote_syslog.init.d]: https://github.com/papertrail/remote_syslog/blob/master/examples/remote_syslog.init.d
+[advanced config]: https://github.com/papertrail/remote_syslog/blob/master/examples/log_files.yml.example.advanced
 [fk]: http://help.github.com/forking/
 [is]: https://github.com/papertrail/remote_syslog/issues/
 [Papertrail]: http://papertrailapp.com/

data/examples/log_files.yml.example.advanced

@@ -0,0 +1,7 @@
+# see README - demonstrates all optional arguments
+files: [/var/log/httpd/access_log, /opt/misc/*]
+hostname: www42   # override OS hostname
+parse_fields: syslog   # predefined regex name or double-quoted regex
+destination:
+  host: logs.papertrailapp.com
+  port: 12345   # optional, defaults to 514

data/lib/remote_syslog.rb

@@ -1,5 +1,5 @@
 module RemoteSyslog
- VERSION = "1.4.1"
+ VERSION = "1.4.2"
 end
 
 require 'remote_syslog/reader'

data/lib/remote_syslog/cli.rb

@@ -49,7 +49,7 @@ module RemoteSyslog
         opts.separator ''
         opts.separator "Example: remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log"
         opts.separator ''
-        opts.separator "Options:"
+        opts.separator "Options (default):"
 
         opts.on("-c", "--configfile PATH", "Path to config (/etc/log_files.yml)") do |v|
           @configfile = File.expand_path(v)
@@ -152,6 +152,7 @@ module RemoteSyslog
     end
 
     def start
+      puts "Watching #{@files.length} files/paths. Sending to #{@dest_host}:#{@dest_port} (#{@tls ? 'TCP/TLS' : 'UDP'})."
       EventMachine.run do
         if @tls
           connection = TlsEndpoint.new(@dest_host, @dest_port)

data/lib/remote_syslog/reader.rb

@@ -53,7 +53,7 @@ module RemoteSyslog
       if @parse_fields
         if message =~ @parse_fields
           packet.hostname = $2 if $2 && $2 != ''
-          packet.tag      = $3 if $3 && $2 != ''
+          packet.tag      = $3 if $3 && $3 != ''
           packet.content  = $4 if $4 && $4 != ''
         end
       end

data/remote_syslog.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'remote_syslog'
-  s.version           = '1.4.1'
-  s.date              = '2011-09-12'
+  s.version           = '1.4.2'
+  s.date              = '2011-12-07'
   s.rubyforge_project = 'remote_syslog'
 
   ## Make sure your summary is short. The description may be as long
@@ -61,7 +61,7 @@ Gem::Specification.new do |s|
     Rakefile
     bin/remote_syslog
     examples/log_files.yml.example
-    examples/log_files.yml.example.syslog
+    examples/log_files.yml.example.advanced
     examples/remote_syslog.init.d
     examples/remote_syslog.supervisor.conf
     lib/remote_syslog.rb

metadata

@@ -1,106 +1,93 @@
---- !ruby/object:Gem::Specification 
+--- !ruby/object:Gem::Specification
 name: remote_syslog
-version: !ruby/object:Gem::Version 
-  prerelease: false
-  segments: 
-  - 1
-  - 4
-  - 1
-  version: 1.4.1
+version: !ruby/object:Gem::Version
+  version: 1.4.2
+  prerelease: 
 platform: ruby
-authors: 
+authors:
 - Troy Davis
 - Eric Lindvall
 autorequire: 
 bindir: bin
 cert_chain: []
-
-date: 2011-09-12 00:00:00 -07:00
-default_executable: remote_syslog
-dependencies: 
-- !ruby/object:Gem::Dependency 
+date: 2011-12-07 00:00:00.000000000Z
+dependencies:
+- !ruby/object:Gem::Dependency
   name: daemons
-  prerelease: false
-  requirement: &id001 !ruby/object:Gem::Requirement 
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        segments: 
-        - 0
-        version: "0"
+  requirement: &70345349040060 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :runtime
-  version_requirements: *id001
-- !ruby/object:Gem::Dependency 
-  name: eventmachine
   prerelease: false
-  requirement: &id002 !ruby/object:Gem::Requirement 
-    requirements: 
+  version_requirements: *70345349040060
+- !ruby/object:Gem::Dependency
+  name: eventmachine
+  requirement: &70345349039500 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
     - - ~>
-      - !ruby/object:Gem::Version 
-        segments: 
-        - 0
-        - 12
-        - 10
+      - !ruby/object:Gem::Version
         version: 0.12.10
   type: :runtime
-  version_requirements: *id002
-- !ruby/object:Gem::Dependency 
-  name: eventmachine-tail
   prerelease: false
-  requirement: &id003 !ruby/object:Gem::Requirement 
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        segments: 
-        - 0
-        version: "0"
+  version_requirements: *70345349039500
+- !ruby/object:Gem::Dependency
+  name: eventmachine-tail
+  requirement: &70345349055440 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :runtime
-  version_requirements: *id003
-- !ruby/object:Gem::Dependency 
-  name: syslog_protocol
   prerelease: false
-  requirement: &id004 !ruby/object:Gem::Requirement 
-    requirements: 
+  version_requirements: *70345349055440
+- !ruby/object:Gem::Dependency
+  name: syslog_protocol
+  requirement: &70345349054880 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
     - - ~>
-      - !ruby/object:Gem::Version 
-        segments: 
-        - 0
-        - 9
-        - 1
+      - !ruby/object:Gem::Version
         version: 0.9.1
   type: :runtime
-  version_requirements: *id004
-- !ruby/object:Gem::Dependency 
-  name: em-resolv-replace
   prerelease: false
-  requirement: &id005 !ruby/object:Gem::Requirement 
-    requirements: 
-    - - ">="
-      - !ruby/object:Gem::Version 
-        segments: 
-        - 0
-        version: "0"
+  version_requirements: *70345349054880
+- !ruby/object:Gem::Dependency
+  name: em-resolv-replace
+  requirement: &70345349054460 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
   type: :runtime
-  version_requirements: *id005
-description: Lightweight daemon to tail one or more log files and transmit UDP syslog messages to a remote syslog host (centralized log aggregation). Generates UDP packets itself instead of depending on a system syslog daemon, so it doesn't affect system-wide logging configuration.
-email: 
+  prerelease: false
+  version_requirements: *70345349054460
+description: Lightweight daemon to tail one or more log files and transmit UDP syslog
+  messages to a remote syslog host (centralized log aggregation). Generates UDP packets
+  itself instead of depending on a system syslog daemon, so it doesn't affect system-wide
+  logging configuration.
+email:
 - troy@sevenscale.com
 - eric@sevenscale.com
-executables: 
+executables:
 - remote_syslog
 extensions: []
-
-extra_rdoc_files: 
+extra_rdoc_files:
 - README.md
 - LICENSE
-files: 
+files:
 - Gemfile
 - LICENSE
 - README.md
 - Rakefile
 - bin/remote_syslog
 - examples/log_files.yml.example
-- examples/log_files.yml.example.syslog
+- examples/log_files.yml.example.advanced
 - examples/remote_syslog.init.d
 - examples/remote_syslog.supervisor.conf
 - lib/remote_syslog.rb
@@ -109,35 +96,30 @@ files:
 - lib/remote_syslog/tls_endpoint.rb
 - lib/remote_syslog/udp_endpoint.rb
 - remote_syslog.gemspec
-has_rdoc: true
 homepage: http://github.com/papertrail/remote_syslog
 licenses: []
-
 post_install_message: 
-rdoc_options: 
+rdoc_options:
 - --charset=UTF-8
-require_paths: 
+require_paths:
 - lib
-required_ruby_version: !ruby/object:Gem::Requirement 
-  requirements: 
-  - - ">="
-    - !ruby/object:Gem::Version 
-      segments: 
-      - 0
-      version: "0"
-required_rubygems_version: !ruby/object:Gem::Requirement 
-  requirements: 
-  - - ">="
-    - !ruby/object:Gem::Version 
-      segments: 
-      - 0
-      version: "0"
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
 requirements: []
-
 rubyforge_project: remote_syslog
-rubygems_version: 1.3.6
+rubygems_version: 1.8.7
 signing_key: 
 specification_version: 2
-summary: Monitor plain text log file(s) for new entries and send to remote syslog collector
+summary: Monitor plain text log file(s) for new entries and send to remote syslog
+  collector
 test_files: []
-

data/examples/log_files.yml.example.syslog

@@ -1,5 +0,0 @@
-files: [/var/log/messages]
-parse_fields: syslog # predefined regex name or double-quoted regex
-destination:
-  host: logs.papertrailapp.com
-  port: 12345   # optional, defaults to 514