netsnmp 0.0.2 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2a043cc79c90239f031b1873089c8a23f50dd5b8
-  data.tar.gz: 05b0baf81082f36cc1a870c3f4f5a87b40041a60
+  metadata.gz: 35229e0d1dd73c4a43a778b6d0e724bbca5ecc12
+  data.tar.gz: f16c14549b5df1408842b8da935b61180751c997
 SHA512:
-  metadata.gz: 63c8a7195237c722b4fb7d0db9f00789ca475c7d080cc8d539f6e1f26adca2482c71d1064925fdd788f95f1f8cac9e89e33c06edc878c22b26be964948af0207
-  data.tar.gz: d30bb03a3a0fc19c34f9afd07512260ea0f4f4bc7f407b5b7c980e9bbb39e166730b34ec5be425b3435270b98e82e002d15b6992ae3834eeb1369bfae805bf19
+  metadata.gz: 45139e881aa85c3350408c9ed59ded3c1ef1c2a418b29fab7953af0a50e056ef235d63280a4958ad537f4b5545e2b5ab0d79c2521b04e0f41ab6a9653f35c47e
+  data.tar.gz: 7621a13aece7a4abcd5b7ec11652a8a39d25ab07ef6206263e8ad378e13aa1d4c397c58fce27e366ab32a845b11e7f7fd043ac2f9a594cf47258fcfc29c94262

data/.coveralls.yml

@@ -0,0 +1 @@
+service_name: travis-ci

data/.travis.yml

@@ -25,12 +25,12 @@ script:
 
 language: ruby
 rvm:
-  - 2.0
   - 2.1
   - 2.2
-  - 2.3.1
+  - 2.3
+  - 2.4
   - ruby-head
-  - jruby-9.1.1.0
+  - jruby-9.1.6.0
   - jruby-head
   - rbx-2
 matrix:
@@ -38,5 +38,5 @@ matrix:
   - rvm: ruby-head
   - rvm: jruby-head
     # to figure out later
-  - rvm: jruby-9.1.1.0
+  - rvm: jruby-9.1.6.0
   - rvm: rbx-2

data/Gemfile

@@ -6,7 +6,11 @@ gem 'coveralls', require: false
 
 group :development do
   gem 'pry'
-  gem 'pry-byebug' 
 end
 
+platforms :mri do
+  gem "pry-byebug", require: false
+  gem "stackprof", require: false
+end
 
+gem "xorcist"

data/README.md

@@ -5,9 +5,7 @@
 [![Code Climate](https://codeclimate.com/github/swisscom/ruby-netsnmp/badges/gpa.svg)](https://codeclimate.com/github/swisscom/ruby-netsnmp)
 [![Docs](http://img.shields.io/badge/yard-docs-blue.svg)](https://www.rubydoc.info/github/swisscom/ruby-netsnmp/master)
 
-The netsnmp gem provides a ruby DSL to handle SNMP queries. It currently uses the net-snmp C library using the FFI interface. 
-
-This gem started as a cleanup from [net-snmp](https://github.com/mixtli/net-snmp) and its follow-up [net-snmp2](https://github.com/jbreeden/net-snmp2), both of which have been mostly inactive for the last year(s).
+The `netsnmp` gem provides a ruby native implementation of the SNMP protocol (v1/2c abd v3).
 
 ## Installation
 
@@ -29,33 +27,38 @@ Or install it yourself as:
 $ gem install netsnmp
 ```
 
-## Why?
+## Features 
 
-You may ask, why not just use the aforementioned? I'll try to sum up the reasons. 
+This gem provides:
 
-* Lack of support for some specific net-snmp versions. 
-* Memory Leaks (both leave the responsibility of cleaning pdus to the user, and this usually creates memory leak when the responses fail). 
-* Lack of support for some ruby > 2.1 GC directives, which makes the VM crash under certain circumstances. 
-* Iffy EventMachine support (they basically send the request pdu, and schedule the reads to the reactor, and not really reacting on the socket handle).
-* Lack of support for NIO4r/Celluloid-IO (the other event loop, besides EM, that counts in the ruby world). 
-* The default sync request calls block the whole VM, making multi-threading a non-factor, forcing you to resort to multiprocess for concurrency. 
+* Implementation in ruby of the SNMP Protocol for v3, v2c and v1 (most notable the rfc3414 and 3826).
+* Client/Manager API with simple interface for get, genext, set and walk.
+* No dependencies.
+* Support for concurrency and evented I/O.
 
+## Why?
 
-## Philosophy
+If you look for snmp gems in ruby toolbox, you'll find a bunch. 
+You may ask, why not just use one of them? 
 
-The main motto of the gem is: API economy. As less moving parts as possible. New features can be discussed and integrated, but they all must abide to this philosophy. 
+Most of them only implement v1 and v2, so if your requirement is to use v3, you're left with only 2 choices: [net-snmp](https://github.com/mixtli/net-snmp) (unmantained since 2013) and its follow-up [net-snmp2](https://github.com/jbreeden/net-snmp2), which started as a fork to fix some bugs left unattended. Both libraries wrap the C netsnmp library using FFI, which leaves them vulnerable to the following bugs (experienced in both libraries):
 
-The main purpose of this gem is: SNMP v3 support. Why? Because it's the only one with authentication and security features integrated. SNMP has been a protocol which historically ignored security for many years, and even when it embraced it, its choices are by today's standards considered half-baked (MD5, SHA-1, shrug). Still, some-security is better than no-security. Also, support for v1 and v2 is something that you can get from other ruby gems (most of them refuse to support v3 altogether).
+* Dependency of specific versions of netsnmp C package. 
+* Memory Leaks.
+* Doesn't work reliable in ruby > 2.0.0-p576, crashing the VM.
+* Network I/O done by the library, thereby blocking the GVL, thereby making all snmp calls block the whole ruby VM.
+  * This means, multi-threading is impossible. 
+  * This means, evented I/O is impossible.
 
-Do one thing and do it well. There's only an interface to interact with an SNMP Agent. Other features like credentials encryption or concurrency are out of the scope of this library, which only guarantees that an SNMP Client is thread-safe and performs ruby-VM-compatible IO. 
+All of these issues are resolved here. 
 
 ## Features
 
-* Client Interface, which supports SNMP 3, 2c, and 1
-* Supports get, set, walk and bulk calls. 
-* Wrappers for eventmachine and celluloid-io
-* Ruby >= 2.0 support
-* net-snmp (C library) >= 5.5 support
+* Client Interface, which supports SNMP v3, v2c, and v1
+* Supports get, getnext, set and walk calls. 
+* Proxy IO object support (for eventmachine/celluloid-io)
+* Ruby >= 2.1 support
+* Pure Ruby (no FFI)
 
 ## Examples
 
@@ -64,13 +67,14 @@ You can use the docker container provided under spec/support to test against the
 ```ruby
 require 'netsnmp'
 
-manager = NETSNMP::Client.new("localhost", port: 33445, username: "simulator",
-                                           auth_password: "auctoritas", auth_protocol: :md5, 
-                                           priv_password: "privatus", priv_protocol: :des,
-                                           context: "a172334d7d97871b72241397f713fa12")
+# example you can test against the docker simulator provided. port attribute might be different. 
+manager = NETSNMP::Client.new(host: "localhost", port: 33445, username: "simulator",
+                              auth_password: "auctoritas", auth_protocol: :md5, 
+                              priv_password: "privatus", priv_protocol: :des,
+                              context: "a172334d7d97871b72241397f713fa12")
 
 # SNMP get
-manager.get("sysName.0") #=> 'tt'
+manager.get(oid: "sysName.0") #=> 'tt'
 
 # SNMP walk
 manager.walk("sysORDescr") do |oid_code, value|
@@ -78,19 +82,13 @@ manager.walk("sysORDescr") do |oid_code, value|
   puts "for #{oid_code}: #{value}"
 end
 
-# SNMP get_bulk
-manager.get_bulk("sysOrDescr") do |oid_code, value|
-  # do something with them  
-  puts "for #{oid_code}: #{value}"
-end
-
 manager.close
 
 # SNMP set
-manager2 = NETSNMP::Client.new("localhost", port: 33445, username: "simulator",
-                                           auth_password: "auctoritas", auth_protocol: :md5, 
-                                           priv_password: "privatus", priv_protocol: :des,
-                                           context: "0886e1397d572377c17c15036a1e6c66")
+manager2 = NETSNMP::Client.new(host: "localhost", port: 33445, username: "simulator",
+                               auth_password: "auctoritas", auth_protocol: :md5, 
+                               priv_password: "privatus", priv_protocol: :des,
+                               context: "0886e1397d572377c17c15036a1e6c66")
 
 # setting to 43, becos yes
 manager2.set("sysUpTimeInstance", 43) 
@@ -98,54 +96,113 @@ manager2.set("sysUpTimeInstance", 43)
 manager2.close
 ```
 
-## Tests
+SNMP v2/v1 examples will be similar (beware of the differences in the initialization attributes). 
 
-This library uses RSpec. The client specs are "integration" tests, in that we communicate with an snmp agent simulator. 
 
-To start the simulator locally, you'll need docker 1.9 or higher (Why 1.9? ```--build-arg``` parameter support was needed for our builds in the CI. You could use a lower version by providing the proxy environment variables in the Dockerfile directly, provided you don't merge these changes to master, thereby exposing your proxy). 
+## Concurrency
 
-```
-> spec/support/start_docker.sh
-```
+In ruby, you are usually adviced not to share IO objects across threads. The same principle applies here to `NETSNMP::Client`: provided you use it within a thread of execution, it should behave safely. So, something like this would be possible:
 
-this builds and starts the docker image in deamonized mode. You can afterwards run your specs:
+```ruby
+general_options = { auth_protocol: ....
+routers.map do |r|
+  Thread.start do 
+    NETSNMP::Client.new(general_options.merge(host: r)) do |cl| 
+      cli.get(oid: "1.6.3.......
 
-```
-> bundle exec rspec
+    end
+  end
+end.each(&:join)
 ```
 
-To stop the image, you can just:
+Evented IO is also supported, in that you can pass a `:proxy` object as an already opened channel of communication to the client. Very important: you have to take care of the lifecycle, as the client will not connect and will not close the object, it will assume no control over it. 
+
+When passing a proxy object, you can omit the `:host` parameter. 
+
+The proxy object will have to be a duck-type implementing `#send`, which is a method receiving the sending PDU payload, and return the payload of the receiving PDU.
+
+Here is a small pseudo-code example: 
+
+```ruby
+# beware, we are inside a warp-speed loop!!!
+general_options = { auth_protocol: ....
+proxy = SpecialUDPImplementation.new(host: router)
+NETSNMP::Client.new(general_options.merge(proxy: proxy)) do |cl|
+  # this get call will eventually #send to the proxy...
+  cli.get(oid: "1.6.3.......
 
+end
+# client isn't usable anymore, but now we must close to proxy
+proxy.close
 ```
-> spec/supoprt/stop_docker.sh
+
+For more information about this subject, the specs test this feature against celluloid-io. An eventmachine could be added, if someone would be kind enough to provide an implementation. 
+
+## Performance
+
+
+### XOR
+
+This library has some workarounds to some missing features in the ruby language, namely the inexistence of a byte array structure. The closest we have is a byte stream presented as a String with ASCII encoding. A method was added to the String class called `#xor` for some operations needed internally. To prevent needless monkey-patches, Refinements have been employed. 
+
+If `#xor` becomes at some point the bottleneck of your usage, this gem has also support for [xorcist](https://github.com/fny/xorcist/). You just have to add it to your Gemfile (or install it in the system):
+
 ```
+# Gemfile
+
+gem 'netsnmp'
+
+# or, in the command line 
+
+$ gem install netsnmp      
+```                        
+
+and `netsnmp` will automatically pick it up. 
 
-## Notes
+## Auth/Priv Key
 
-This library provides support for C net-snmp 5.5 or higher, as this has been considered the most stable SNMP v3 implementation.
+If you'll use this gem often with SNMP v3 and auth/priv security level enabled, you'll have that funny feeling that everything could be a bit faster. Well, this is basically because the true performance bottleneck of this gem is the generation of the auth and pass keys used for authorization and encryption. Although this is a one-time thing for each client, its lag will be noticeable if you're running on > 100 hosts.
 
-To install it in your environment, just use your package manager:
+There is a recommended work-around, but this is only usable **if you are using the same user/authpass/privpass on all the hosts!!!**. Use this with care, then:
 
+```ruby
+$shared_security_parameters = NETSNMP::SecurityParameters.new(security_level: :authpriv, username: "mustermann", 
+                                                              auth_protocol: :md5, priv_protocol: :aes, ....
+# this will eager-load the auth/priv_key
+...
+
+# over 9000 routers are running on this event loop!!! this is just one!
+NETSNMP::Client.new(share_options.merge(proxy: router_proxy, security_parameters: $shared_security_parameters.dup).new do |cl|
+  cli.get(oid:  .....
+end
 ```
-# on OSX
-> brew install net-snmp  
 
-# Linux
-# centOS
-> sudo yum install net-snmp
-# Ubuntu
-> sudo apt-get install net-snmp
-> sudo apt-get install libsnmp-dev
+## OpenSSL
+
+All encoding/decoding/encryption/decryption/digests are done using `openssl`, which is (still) a part of the standard library. If at some point `openssl` is removed and not specifically distributed, you'll have to install it yourself. Hopefully this will never happen.
+
+
+## Tests
+
+This library uses RSpec. The client specs are "integration" tests, in that we communicate with an snmp agent simulator. 
+
+To start the simulator locally, you'll need docker 1.9 or higher (Why 1.9? ```--build-arg``` parameter support was needed for our builds in the CI. You could use a lower version by providing the proxy environment variables in the Dockerfile directly, provided you don't merge these changes to master, thereby exposing your proxy). 
+
+```
+> spec/support/start_docker.sh
 ```
 
-Ubuntu doesn't come with the default mibs, you might have to download the mibs downloader package:
+this builds and starts the docker image in deamonized mode. You can afterwards run your specs:
 
 ```
-> sudo apt-get install -y snmp-mibs-downloader
-> sudo download-mibs
+> bundle exec rspec
 ```
 
-TODO: on Windows(?)
+To stop the image, you can just:
+
+```
+> spec/supoprt/stop_docker.sh
+```
 
 ## Contributing
 
@@ -156,7 +213,11 @@ TODO: on Windows(?)
 
 ## TODO
 
-There are a few features still to be added, like:
+There are some features which this gem doesn't support. It was built to provide a client (or manager, in SNMP language) implementation only, and the requirements were fulfilled. However, these notable misses will stand-out:
+
+* No MIB support (you can only work with OIDs)
+* No server (Agent, in SNMP-ish) implementation.
+* No getbulk support. 
 
-* [JRuby Support](https://github.com/celluloid/nio4r/issues/94)
+So if you like the gem, but would rather have these features implemented, please help by sending us a PR and we'll gladly review it.
 

data/lib/netsnmp.rb

@@ -1,16 +1,72 @@
-require 'netsnmp/version'
+# frozen_string_literal: true
+require "netsnmp/version"
+require "openssl"
+require "io/wait"
+require "securerandom"
+
 
 # core structures
-require 'netsnmp/core'
+
+begin
+  require "xorcist"
+  require "xorcist/refinements"
+  NETSNMP::StringExtensions = Xorcist::Refinements
+rescue LoadError
+  # "no xorcist"
+  module NETSNMP
+    module StringExtensions
+      refine String do
+        # Bitwise XOR operator for the String class
+        def xor( other )
+          b1 = self.unpack("C*")
+          return b1 if !other
+      
+          b2 = other.unpack("C*")
+          longest = [b1.length,b2.length].max
+          b1 = [0]*(longest-b1.length) + b1
+          b2 = [0]*(longest-b2.length) + b2
+          b1.zip(b2).map{ |a,b| a^b }.pack("C*")
+        end
+      end
+    end
+  end
+end
+
 
 module NETSNMP
-  # @return [String] the version of the netsnmp C library
-  def self.version ; Core.version ; end
+  def self.set_debug(io)
+    @debug_output = io
+  end
+
+  def self.debug(&blk)
+    @debug_output << blk.call + "\n" if @debug_output
+  end
+
+  unless defined?(Hexdump) # support the hexdump gem
+    module Hexdump
+      def self.dump(data, width: 8)
+        pairs = data.unpack("H*").first.scan(/.{4}/)
+        pairs.each_slice(width).map do |row|
+          row.join(" ")
+        end.join("\n")
+      end
+    end
+  end
 end
 
-require 'netsnmp/errors'
-require 'netsnmp/varbind'
-require 'netsnmp/oid'
-require 'netsnmp/pdu'
-require 'netsnmp/session'
-require 'netsnmp/client'
+
+require "netsnmp/errors"
+
+require "netsnmp/oid"
+require "netsnmp/varbind"
+require "netsnmp/pdu"
+require "netsnmp/session"
+
+require "netsnmp/scoped_pdu"
+require "netsnmp/v3_session"
+require "netsnmp/security_parameters"
+require "netsnmp/message"
+require "netsnmp/encryption/des"
+require "netsnmp/encryption/aes"
+
+require "netsnmp/client"

data/lib/netsnmp/client.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module NETSNMP
   # Main Entity, provides the user-facing API to communicate with SNMP Agents
   #
@@ -7,83 +8,90 @@ module NETSNMP
   #
   #
   class Client
-    
-    # @param [String] hostname the hostname of the agent
-    # @param [Hash] options the set of options to open the session.
+    RETRIES = 5
+
+    # @param [Hash] options the options to needed to enable the SNMP client.
+    # @option options [String, Integer, nil] :version the version of the protocol (defaults to 3).
+    #    also accepts common known declarations like :v3, "v2c", etc
+    # @option options [Integer] :retries number of retries for each failed PDU (after which it raise timeout error. Defaults to {RETRIES} retries)
+    # @yield [client] the instantiated client, after which it closes it for use. 
+    # @example Yielding a clinet
+    #    NETSNMP::Client.new(host: "241.232.22.12") do |client|
+    #      puts client.get(oid: "1.3.6.1.2.1.1.5.0")
+    #    end
     #
-    # @see Session#initialize
-    def initialize(hostname, options)
-      @session ||= Session.new(hostname, options)
+    def initialize(**options)
+      version = options[:version]
+      version = case version 
+        when Integer then version # assume the use know what he's doing
+        when /v?1/ then 0 
+        when /v?2c?/ then 1 
+        when /v?3/, nil then 3
+      end
+
+      @retries = options.fetch(:retries, RETRIES)
+      @session ||= version == 3 ? V3Session.new(options) : Session.new(options)
+      if block_given?
+        begin
+          yield self
+        ensure
+          close
+        end
+      end
     end
 
-    # @see Session#close
+    # Closes the inner section 
     def close
       @session.close
     end
 
     # Performs an SNMP GET Request
-    # 
-    # @param [OID, String] oid_code the oid to get
-    # @param [Hash] options the varbind options (see Varbind)
-    # @option options [true, false] :response_pdu if true, the method returns a PDU
     #
-    # @return [String] the value for the oid
+    # @see {NETSNMP::Varbind#new}
     #
-    def get(oid_code, **options)
-      request_pdu = RequestPDU.build(:get)
-      oid = oid_code.is_a?(OID) ? oid_code : OID.new(oid_code)
-      request_pdu.add_varbind(oid, options)
-      yield request_pdu if block_given? 
-      response_pdu = @session.send(request_pdu)
-      case options[:response_type] 
-        when :pdu then response_pdu
-        else response_pdu.value
-      end
+    def get(oid_opts)
+      request = @session.build_pdu(:get, oid_opts)
+      response = handle_retries { @session.send(request) }
+      yield response if block_given?
+      response.varbinds.first.value
     end
 
     # Performs an SNMP GETNEXT Request
     # 
-    # @param [OID, String] oid_code the oid to get
-    # @param [Hash] options the varbind options (see Varbind)
-    # @option options [true, false] :response_pdu if true, the method returns a PDU
-    #
-    # @return [String] the value for the next oid
-    # 
-    # @note this method is used as a sub-routine for the walk
+    # @see {NETSNMP::Varbind#new}
     #
-    def get_next(oid_code, **options)
-      request_pdu = RequestPDU.build(:getnext)
-      oid = oid_code.is_a?(OID) ? oid_code : OID.new(oid_code)
-      request_pdu.add_varbind(oid, options)
-      yield request_pdu if block_given? 
-      response_pdu = @session.send(request_pdu)
-      case options[:response_type] 
-        when :pdu then response_pdu
-        else response_pdu.value
-      end
+    def get_next(oid_opts)
+      request = @session.build_pdu(:getnext, oid_opts)
+      response = handle_retries { @session.send(request) }
+      yield response if block_given?
+      varbind = response.varbinds.first
+      [varbind.oid, varbind.value]
     end
 
     # Perform a SNMP Walk (issues multiple subsequent GENEXT requests within the subtree rooted on an OID)
     #
-    # @param [OID, String] oid_code the root oid from the subtree
-    # @param [Hash] options the varbind options 
+    # @param [String] oid the root oid from the subtree
     #
     # @return [Enumerator] the enumerator-collection of the oid-value pairs
     #
-    def walk(oid_code, **options)
-      options[:response_type] = :pdu
-      walkoid = oid_code.is_a?(OID) ? oid_code : OID.new(oid_code)
+    def walk(oid: )
+      walkoid = oid
       Enumerator.new do |y|
-        code = walkoid.code
+        code = walkoid
+        first_response_code = nil
         catch(:walk) do
           loop do
-            response_pdu = get_next(code, options)
-            response_pdu.varbinds.each do |varbind|
-              code = varbind.oid_code
-              if !walkoid.parent_of?(code) or varbind.value.eql?(:endofmibview)
-                throw(:walk)             
-              else
-                y << [code, varbind.value]
+            get_next(oid: code) do |response|
+              response.varbinds.each do |varbind|
+                code = varbind.oid
+                if !OID.parent?(walkoid, code) or 
+                    varbind.value.eql?(:endofmibview) or
+                    code == first_response_code
+                  throw(:walk)             
+                else
+                  y << [code, varbind.value]
+                end
+                first_response_code ||= code
               end
             end
           end
@@ -93,39 +101,49 @@ module NETSNMP
 
     # Perform a SNMP GETBULK Request (performs multiple GETNEXT)
     #
-    # @param [OID, String] oid_code the first oid
+    # @param [String] oid the first oid
     # @param [Hash] options the varbind options 
     # @option options [Integer] :errstat sets the number of objects expected for the getnext instance
     # @option options [Integer] :errindex number of objects repeating for all the repeating IODs. 
     #
     # @return [Enumerator] the enumerator-collection of the oid-value pairs
     #
-    def get_bulk(oid_code, **options)
-      request_pdu = RequestPDU.build(:getbulk)
-      request_pdu[:errstat]  = options.delete(:non_repeaters) || 0
-      request_pdu[:errindex] = options.delete(:max_repetitions) || 10
-      request_pdu.add_varbind(OID.new(oid_code), options)
-      yield request_pdu if block_given? 
-      response_pdu = @session.send(request_pdu)
-      Enumerator.new do |y|
-        response_pdu.varbinds.each do |varbind|
-          y << [ varbind.oid_code, varbind.value ]
-        end
-      end
-    end
+    #def get_bulk(oid)
+    #  request = @session.build_pdu(:getbulk, *oids)
+    #  request[:error_status]  = options.delete(:non_repeaters) || 0
+    #  request[:error_index] = options.delete(:max_repetitions) || 10
+    #  response = @session.send(request)
+    #  Enumerator.new do |y|
+    #    response.varbinds.each do |varbind|
+    #      y << [ varbind.oid, varbind.value ]
+    #    end
+    #  end
+    #end
 
     # Perform a SNMP SET Request
     #
-    # @param [OID, String] oid_code the oid to update
-    # @param [Hash] options the varbind options 
-    # @option options [Object] :value value to update the oid with. 
+    # @see {NETSNMP::Varbind#new}
     #
-    def set(oid_code, **options)
-      request_pdu = PDU.build(:set)
-      request_pdu.add_varbind(OID.new(oid_code), options)
-      yield request_pdu if block_given? 
-      response_pdu = @session.send(request_pdu)
-      response_pdu.value
+    def set(oid_opts)
+      request = @session.build_pdu(:set, oid_opts)
+      response = handle_retries { @session.send(request) }
+      yield response if block_given? 
+      response.varbinds.map(&:value)
+    end
+
+
+    private
+
+    # Handles timeout errors by reissuing the same pdu until it runs out or retries. 
+    def handle_retries
+      retries = @retries
+      begin
+        yield
+      rescue Timeout::Error => e
+        raise e if retries == 0
+        retries -= 1
+        retry
+      end
     end
   end
 end