remote_syslog 1.4.2 → 1.5.0

data/README.md

@@ -1,22 +1,22 @@
 # remote_syslog Ruby daemon & sender
 
-Lightweight Ruby daemon to tail one or more log files and transmit UDP syslog 
+Lightweight Ruby daemon to tail one or more log files and transmit UDP syslog
 messages to a remote syslog host (centralized log aggregation).
 
-remote_syslog generates UDP packets itself instead of depending on a system 
-syslog daemon, so its configuration doesn't affect system-wide 
+remote_syslog generates UDP packets itself instead of depending on a system
+syslog daemon, so its configuration doesn't affect system-wide
 logging - syslog is just the transport.
 
 Uses:
 
 * collecting logs from servers & daemons which don't natively support syslog
-* when reconfiguring the system logger is less convenient than a 
+* when reconfiguring the system logger is less convenient than a
 purpose-built daemon (e.g., automated app deployments)
 * aggregating files not generated by daemons (e.g., package manager logs)
 
 The library can also be used to generate one-off log messages from Ruby code.
 
-Tested with the hosted log management service [Papertrail] and should work for 
+Tested with the hosted log management service [Papertrail] and should work for
 transmitting to any syslog server.
 
 
@@ -26,19 +26,19 @@ Install the gem, which includes a binary called "remote_syslog":
 
     $ [sudo] gem install remote_syslog
 
-Optionally, create a log_files.yml with the log file paths to read and the 
-host/port to log to (see examples/[log_files.yml.example][sample config]). These can also be 
+Optionally, create a log_files.yml with the log file paths to read and the
+host/port to log to (see examples/[log_files.yml.example][sample config]). These can also be
 specified as command-line arguments (below).
 
 
 ## Usage
 
     $ remote_syslog -h
-    Usage:   remote_syslog [options] <path to add'l log 1> .. <path to add'l log n>
+    Usage: remote_syslog [options] [<logfile>...]
 
     Example: remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log
 
-    Options (default):
+    Options:
         -c, --configfile PATH            Path to config (/etc/log_files.yml)
         -d, --dest-host HOSTNAME         Destination syslog hostname or IP (logs.papertrailapp.com)
         -p, --dest-port PORT             Destination syslog port (514)
@@ -52,7 +52,7 @@ specified as command-line arguments (below).
             --strip-color                Strip color codes
             --tls                        Connect via TCP with TLS
         -h, --help                       Show this message
-    
+
 
 ## Example
 
@@ -60,7 +60,7 @@ Typical:
 
     $ remote_syslog
 
-Daemonize and collect messages from files listed in `./config/logs.yml` as 
+Daemonize and collect messages from files listed in `./config/logs.yml` as
 well as the file `/var/log/mysqld.log`. Send to port `logs.papertrailapp.com:12345`:
 
     $ remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log
@@ -71,7 +71,7 @@ to `a.server.com:514`:
 
     $ remote_syslog -D -d a.server.com -f local0 -P /tmp /var/log/mysqld.log
 
-remote_syslog will daemonize by default. A sample init file is in the gem as 
+remote_syslog will daemonize by default. A sample init file is in the gem as
 [remote_syslog.init.d]. You may be able to:
 
     $ cp examples/remote_syslog.init.d /etc/init.d/remote_syslog
@@ -90,21 +90,25 @@ By default, the gem looks for a configuration in /etc/log_files.yml.
 The gem comes with a [sample config].  Optionally:
 
     $ cp examples/log_files.yml.example /etc/log_files.yml
-    
-log_files.yml has filenames to log from (as an array) and hostname and port 
-to log to (as a hash). Wildcards are supported using * and standard shell 
-globbing. Filenames given on the command line are additive to those in 
+
+log_files.yml has filenames to log from (as an array) and hostname and port
+to log to (as a hash). Wildcards are supported using * and standard shell
+globbing. Filenames given on the command line are additive to those in
 the config file.
 
-Only 1 destination server is supported; the command-line argument wins. 
+Only 1 destination server is supported; the command-line argument wins.
 
-    files: [/var/log/httpd/access_log, /var/log/httpd/error_log, /var/log/mysqld.log, /var/run/mysqld/mysqld-slow.log]
+    files:
+     - /var/log/httpd/access_log
+     - /var/log/httpd/error_log
+     - /var/log/mysqld.log
+     - /var/run/mysqld/mysqld-slow.log
     destination:
       host: logs.papertrailapp.com
       port: 12345
 
-remote_syslog sends the name of the file without a path ("mysqld.log") as 
-the syslog tag (program name). RFCs 3164 and 5424 limit the tag to 32 
+remote_syslog sends the name of the file without a path ("mysqld.log") as
+the syslog tag (program name). RFCs 3164 and 5424 limit the tag to 32
 characters. Longer filenames are truncated to 32 characters.
 
 ## Advanced Configuration (Optional)
@@ -117,9 +121,24 @@ Provide `--hostname somehostname` or use the `hostname` configuration option:
 
     hostname: somehostname
 
+### Verify server certificate
+
+Provide the public key for the remote host when using TLS:
+
+    ssl_server_cert: syslog.crt
+
+
+### Use a client certificate
+
+Provide a client certificate when connecting via TLS:
+
+    ssl_client_cert_chain: syslog_client.crt
+    ssl_client_private_key: syslog_client.key
+
+
 ### Multiple instances
 
-Run multiple instances to support more than one message-specific file format 
+Run multiple instances to support more than one message-specific file format
 or to specify unique syslog hostnames.
 
 To do that, provide an alternate PID filename as a command-line option
@@ -127,49 +146,50 @@ to the additional instance(s). For example:
 
     --pid-file remote_syslog_2.pid
 
+
 ### Parse fields from log messages
 
-Rarely needed. Usually only used when remote_syslog is watching files 
+Rarely needed. Usually only used when remote_syslog is watching files
 generated by syslogd (rather than by apps), like ``/var/log/messages``.
 
-remote_syslog can parse the program and hostname from the log line. When one 
-file contains logs from multiple programs (like with syslog), the log line 
-may include text that is not part of the log message, like a timestamp, 
-hostname, or program name. remote_syslog will extract those and use them in 
+remote_syslog can parse the program and hostname from the log line. When one
+file contains logs from multiple programs (like with syslog), the log line
+may include text that is not part of the log message, like a timestamp,
+hostname, or program name. remote_syslog will extract those and use them in
 the corresponding syslog packet fields.
 
-To do that, use the config file option `parse_fields` with the name of a 
+To do that, use the config file option `parse_fields` with the name of a
 format supported by remote_syslog, or your own regex. Included format names
 are `syslog` and `rfc3339`. For example:
 
     parse_fields: syslog
 
-The included `syslog` format uses the regex `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)` 
+The included `syslog` format uses the regex `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)`
 to parse standard syslog lines like this:
 
     Jul 18 08:25:08 hostname programname[1234]: The log message
 
-The included `rfc3339` format uses the regex `(\S+) (\S+) ([^: ]+):? (.*)` to 
+The included `rfc3339` format uses the regex `(\S+) (\S+) ([^: ]+):? (.*)` to
 parse syslog lines with high-precision RFC 3339 timestamps, like this:
 
     2011-07-16T08:25:08.651413-07:00 hostname programname[1234]: The log message
 
-To parse a format other than those, provide your own regex. It should include 
-4 backreferences to parse, in order: timestamp, system name, program name, 
+To parse a format other than those, provide your own regex. It should include
+4 backreferences to parse, in order: timestamp, system name, program name,
 message.
 
-Match and return empty strings for any empty positions where the log line 
+Match and return empty strings for any empty positions where the log line
 doesn't provide a value. For example, given the log message:
 
     something-meaningless The log message
 
 One could use a regex to ignore "something-meaningless" (and not to extract
-a program or hostname). To ignore that prefix and return 3 empty values 
+a program or hostname). To ignore that prefix and return 3 empty values
 then the log message, use parse_fields with this regex:
 
     parse_fields: "something-meaningless ()()()(.*)"
 
-Per-file regexes are not supported. Run multiple instances with different 
+Per-file regexes are not supported. Run multiple instances with different
 config files.
 
 

data/examples/log_files.yml.example

@@ -1,4 +1,9 @@
-files: [/var/log/httpd/access_log, /var/log/httpd/error_log, /opt/misc/*, /var/log/mysqld.log, /var/run/mysqld/mysqld-slow.log]
+files: 
+  - /var/log/httpd/access_log
+  - /var/log/httpd/error_log
+  - /opt/misc/*.log
+  - /var/log/mysqld.log
+  - /var/run/mysqld/mysqld-slow.log
 destination:
   host: logs.papertrailapp.com
   port: 12345   # optional, defaults to 514

data/lib/remote_syslog.rb

@@ -1,5 +1,5 @@
 module RemoteSyslog
- VERSION = "1.4.2"
+ VERSION = "1.5.0"
 end
 
 require 'remote_syslog/reader'

data/lib/remote_syslog/cli.rb

@@ -45,11 +45,11 @@ module RemoteSyslog
 
     def parse
       op = OptionParser.new do |opts|
-        opts.banner = "Usage:   remote_syslog [options] <path to add'l log 1> .. <path to add'l log n>"
+        opts.banner = "Usage: remote_syslog [options] [<logfile>...]"
         opts.separator ''
         opts.separator "Example: remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log"
         opts.separator ''
-        opts.separator "Options (default):"
+        opts.separator "Options:"
 
         opts.on("-c", "--configfile PATH", "Path to config (/etc/log_files.yml)") do |v|
           @configfile = File.expand_path(v)
@@ -117,9 +117,7 @@ module RemoteSyslog
 
     def parse_config
       if File.exist?(@configfile)
-        config = open(@configfile) do |f|
-          YAML.load(f)
-        end
+        config = YAML.load_file(@configfile)
 
         @files += Array(config['files'])
 
@@ -135,6 +133,10 @@ module RemoteSyslog
           @hostname = config['hostname']
         end
 
+        @server_cert        = config['ssl_server_cert']
+        @client_cert_chain  = config['ssl_client_cert_chain']
+        @client_private_key = config['ssl_client_private_key']
+
         if config['parse_fields']
           @parse_fields = FIELD_REGEXES[config['parse_fields']] || Regexp.new(config['parse_fields'])
         end
@@ -142,6 +144,8 @@ module RemoteSyslog
     end
 
     def run
+      puts "Watching #{@files.length} files/paths. Sending to #{@dest_host}:#{@dest_port} (#{@tls ? 'TCP/TLS' : 'UDP'})."
+
       if @no_detach
         start
       else
@@ -152,10 +156,12 @@ module RemoteSyslog
     end
 
     def start
-      puts "Watching #{@files.length} files/paths. Sending to #{@dest_host}:#{@dest_port} (#{@tls ? 'TCP/TLS' : 'UDP'})."
       EventMachine.run do
         if @tls
-          connection = TlsEndpoint.new(@dest_host, @dest_port)
+          connection = TlsEndpoint.new(@dest_host, @dest_port,
+            :client_cert_chain => @client_cert_chain,
+            :client_private_key => @client_private_key,
+            :server_cert => @server_cert)
         else
           connection = UdpEndpoint.new(@dest_host, @dest_port)
         end

data/lib/remote_syslog/tls_endpoint.rb

@@ -3,12 +3,22 @@ module RemoteSyslog
     class Handler < EventMachine::Connection
       def initialize(endpoint)
         @endpoint = endpoint
-        @endpoint.connection = self
         super()
       end
 
       def connection_completed
-        start_tls
+        start_tls(:verify_peer => @endpoint.server_cert != nil,
+          :cert_chain_file => @endpoint.client_cert_chain,
+          :private_key_file => @endpoint.client_private_key)
+      end
+
+      def ssl_verify_peer(peer_cert)
+        peer_cert = OpenSSL::X509::Certificate.new(peer_cert)
+        peer_cert.verify(@endpoint.server_cert.public_key)
+      end
+
+      def ssl_handshake_completed
+        @endpoint.connection = self
       end
 
       def unbind
@@ -17,10 +27,17 @@ module RemoteSyslog
     end
 
     attr_accessor :connection
+    attr_reader :server_cert, :client_cert_chain, :client_private_key
 
-    def initialize(address, port)
-      @address = address
-      @port    = port.to_i
+    def initialize(address, port, options = {})
+      @address            = address
+      @port               = port.to_i
+      @client_cert_chain  = options[:client_cert_chain]
+      @client_private_key = options[:client_private_key]
+
+      if options[:server_cert]
+        @server_cert = OpenSSL::X509::Certificate.new(File.read(options[:server_cert]))
+      end
 
       # Try to resolve the address
       resolve_address

data/remote_syslog.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'remote_syslog'
-  s.version           = '1.4.2'
-  s.date              = '2011-12-07'
+  s.version           = '1.5.0'
+  s.date              = '2011-12-30'
   s.rubyforge_project = 'remote_syslog'
 
   ## Make sure your summary is short. The description may be as long

metadata

@@ -1,86 +1,99 @@
---- !ruby/object:Gem::Specification
+--- !ruby/object:Gem::Specification 
 name: remote_syslog
-version: !ruby/object:Gem::Version
-  version: 1.4.2
-  prerelease: 
+version: !ruby/object:Gem::Version 
+  prerelease: false
+  segments: 
+  - 1
+  - 5
+  - 0
+  version: 1.5.0
 platform: ruby
-authors:
+authors: 
 - Troy Davis
 - Eric Lindvall
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2011-12-07 00:00:00.000000000Z
-dependencies:
-- !ruby/object:Gem::Dependency
+
+date: 2011-12-30 00:00:00 -08:00
+default_executable: remote_syslog
+dependencies: 
+- !ruby/object:Gem::Dependency 
   name: daemons
-  requirement: &70345349040060 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
   prerelease: false
-  version_requirements: *70345349040060
-- !ruby/object:Gem::Dependency
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
   name: eventmachine
-  requirement: &70345349039500 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
+  prerelease: false
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    requirements: 
     - - ~>
-      - !ruby/object:Gem::Version
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        - 12
+        - 10
         version: 0.12.10
   type: :runtime
-  prerelease: false
-  version_requirements: *70345349039500
-- !ruby/object:Gem::Dependency
+  version_requirements: *id002
+- !ruby/object:Gem::Dependency 
   name: eventmachine-tail
-  requirement: &70345349055440 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
   prerelease: false
-  version_requirements: *70345349055440
-- !ruby/object:Gem::Dependency
+  requirement: &id003 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id003
+- !ruby/object:Gem::Dependency 
   name: syslog_protocol
-  requirement: &70345349054880 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
+  prerelease: false
+  requirement: &id004 !ruby/object:Gem::Requirement 
+    requirements: 
     - - ~>
-      - !ruby/object:Gem::Version
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        - 9
+        - 1
         version: 0.9.1
   type: :runtime
-  prerelease: false
-  version_requirements: *70345349054880
-- !ruby/object:Gem::Dependency
+  version_requirements: *id004
+- !ruby/object:Gem::Dependency 
   name: em-resolv-replace
-  requirement: &70345349054460 !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
   prerelease: false
-  version_requirements: *70345349054460
-description: Lightweight daemon to tail one or more log files and transmit UDP syslog
-  messages to a remote syslog host (centralized log aggregation). Generates UDP packets
-  itself instead of depending on a system syslog daemon, so it doesn't affect system-wide
-  logging configuration.
-email:
+  requirement: &id005 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id005
+description: Lightweight daemon to tail one or more log files and transmit UDP syslog messages to a remote syslog host (centralized log aggregation). Generates UDP packets itself instead of depending on a system syslog daemon, so it doesn't affect system-wide logging configuration.
+email: 
 - troy@sevenscale.com
 - eric@sevenscale.com
-executables:
+executables: 
 - remote_syslog
 extensions: []
-extra_rdoc_files:
+
+extra_rdoc_files: 
 - README.md
 - LICENSE
-files:
+files: 
 - Gemfile
 - LICENSE
 - README.md
@@ -96,30 +109,35 @@ files:
 - lib/remote_syslog/tls_endpoint.rb
 - lib/remote_syslog/udp_endpoint.rb
 - remote_syslog.gemspec
+has_rdoc: true
 homepage: http://github.com/papertrail/remote_syslog
 licenses: []
+
 post_install_message: 
-rdoc_options:
+rdoc_options: 
 - --charset=UTF-8
-require_paths:
+require_paths: 
 - lib
-required_ruby_version: !ruby/object:Gem::Requirement
-  none: false
-  requirements:
-  - - ! '>='
-    - !ruby/object:Gem::Version
-      version: '0'
-required_rubygems_version: !ruby/object:Gem::Requirement
-  none: false
-  requirements:
-  - - ! '>='
-    - !ruby/object:Gem::Version
-      version: '0'
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
 requirements: []
+
 rubyforge_project: remote_syslog
-rubygems_version: 1.8.7
+rubygems_version: 1.3.6
 signing_key: 
 specification_version: 2
-summary: Monitor plain text log file(s) for new entries and send to remote syslog
-  collector
+summary: Monitor plain text log file(s) for new entries and send to remote syslog collector
 test_files: []
+