antfarm-core 0.5.0.beta1

data/README.md

@@ -0,0 +1,190 @@
+# ANTFARM-CORE
+
+ANTFARM (Advanced Network Toolkit For Assessments and Remote Mapping) is a
+passive network mapping application that utilizes output from existing network
+examination tools to populate its OSI-modeled database. This data can then be
+used to form a ‘picture’ of the network being analyzed.
+
+ANTFARM can also be described as a data fusion tool that does not directly
+interact with the network. The analyst can use a variety of passive or active
+data gathering techniques, the outputs of which are loaded into ANTFARM and
+incorporated into the network map. Data gathering can be limited to completely
+passive techniques when minimizing the risk of disrupting the operational
+network is a concern.
+
+This library implements the core ANTFARM functionality, which mainly facilitates
+creating and interacting with the relational database that holds and correlates
+network data as it is parsed. This library is not meant to stand alone, but
+rather be part of a larger application needing ANTFARM functionality. Please
+see the ANTFARM (as opposed to the ANTFARM-CORE) library if you are looking for
+the command-line application.
+
+## STATUS OF BETA RELEASE
+
+Please note that not all of the database models and plugins available in
+version 0.4.0 are available in the beta release of version 0.5.0. The 'beta'
+status will be removed once all the functionality available in version 0.4.0 is
+implemented in version 0.5.0.
+
+The database models not yet available in the beta release of version 0.5.0 are:
+
+* Action
+* DnsEntry
+* OperatingSystem
+* PrivateNetwork
+* Service
+* Traffic
+
+Note that in order to install a pre-release version of a gem (such as the beta
+version of ANTFARM 0.5.0), you must specify the --pre-release option when using
+'gem install' along with the full version name (i.e. --version 0.5.0.beta).
+
+## HOW IT WORKS
+
+At the center of the ANTFARM-CORE library is a boot-strapping and initialization
+process very similar to the one used in Rails applications. The boot-strapping
+and initialization process sets the root directory, the environment to use (used
+by the database and logging features), the log level to use, and loads in all
+the database models (see below).
+
+DataMapper is used as the ORM for interacting with the database, and models
+exist for the following database tables:
+
+* Node
+* LayerTwoInterface
+* EthernetInterface
+* LayerThreeInterface
+* IpInterface
+* LayerThreeNetwork
+* IpNetwork
+
+These models live in the Antfarm::Model namespace.
+
+A framework is provided to facilitate interaction with plugins and manipulation
+of the database.
+
+## THINGS TO KNOW
+
+The ANTFARM environment and log settings can (and should) be set via the
+described environment variables below as long as they are set before the
+config/environment.rb file is loaded.
+
+    ENV['ANTFARM_ENV'] = 'foo'
+    ENV['ANTFARM_LOG_LEVEL'] = 'debug'
+
+When ANTFARM is boot-strapped, it will check to see if a .antfarm directory
+exists in the home directory of the current user and will create it if not.
+This is where application-specific data is stored, like default environment
+and log level settings, database settings, SQLite3 databases (if used), and
+log files. Custom user plugins can also be placed in the .antfarm directory
+and they will be recognized by the plugins library.
+
+## DATABASE SETTINGS
+
+Right now, only SQLite3 is supported. As such, it is the default. Future plans
+include supporting Postgres as well, in which case different databases can be
+configured for different environments via the default settings in the .antfarm
+directory.
+
+## PLUGINS
+
+Detailed information for each plugin is provided via the ANTFARM-PLUGINS man
+page (`gem man antfarm-plugins`). Plugins included in the core library are
+located in the 'lib/antfarm/plugins/' directory, and custom plugins created by
+a user would/should be located in the '~/.antfarm/plugins' directory.
+
+## HOW TO WRITE A PLUGIN
+
+The requirements for a plugin are as follows:
+
+* Plugin must belong to the Antfarm::Plugin namespace
+* Below the Antfarm::Plugin namespace, namespacing must follow the directory
+  structure of the location of the plugin
+* Plugin must include the Antfarm::Plugin module
+* Plugin must provide a hash that describes the plugin and an array of hashes
+  that describe possible plugin options to 'super' in the constructor
+** Required description options are :name, :desc, and :author
+** Required parameter options are :name, :desc, :type, :default and :required
+* Plugin must implement a 'run' method that accepts a single hash parameter
+** The single hash parameter will contain options provided as described in the
+   constructor
+
+Here is a very simple example plugin located at 'plugins/custom/foo-bar.rb':
+
+    module Antfarm
+      module Plugin
+        module Custom
+          class FooBar
+            include Antfarm::Plugin
+
+            def initialize
+              super( { :name => 'Foo Bar Plugin',
+                       :desc => 'This plugin does nothing',
+                       :author => 'Me <me@you.com>' },
+                    [{ :name => :input_file,
+                       :desc => 'File that has data in it',
+                       :type => String,
+                       :required => true },
+                     { :name => :use,
+                       :desc => 'To use or not to use' }
+                   ])
+            end
+
+            def run(options)
+              # options[:input_file] will contain a string
+              # options[:use] will either be true or false, depending on whether or
+              # not the user provided the flag
+              
+              # TODO: do something!
+              # Database models can be used like so:
+              #   Antfarm::Model::IpInterface.create :address => 'w.x.y.z'
+            end
+          end
+        end
+      end
+    end
+
+Note that for optional parameters, if a type is not provided it is assumed to be
+a flag (true if the flag is provided, false if not). Obviously the default will
+be false and it is not required.
+
+## VERSIONING INFORMATION
+
+This project uses the major/minor/bugfix method of versioning. It has yet to
+reach a 1.x.x status yet because the API is still in flux. When new plugins are
+officially released, the minor version number will be incremented.
+
+## DISCLAIMER
+
+While the ANTFARM-CORE library is completely passive (it does not have any
+built-in means of gathering data directly from devices or networks), network
+admin tools that users of ANTFARM may choose to gather data with may or may not
+be passive. The authors of ANTFARM hold no responsibility in how users decide to
+gather data they wish to feed into ANTFARM.
+
+## COPYRIGHT
+
+Copyright (2008-2010) Sandia Corporation. Under the terms of Contract
+DE-AC04-94AL85000 with Sandia Corporation, the U.S. Government retains certain
+rights in this software.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, distribute with modifications,
+sublicense, and/or sell copies of the Software, and to permit persons to whom
+the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ABOVE COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+Except as contained in this notice, the name(s) of the above copyright holders
+shall not be used in advertising or otherwise to promote the sale, use or other
+dealings in this Software without prior written authorization.

data/config/boot.rb

@@ -0,0 +1,40 @@
+# Copyright (2008) Sandia Corporation.
+# Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
+# the U.S. Government retains certain rights in this software.
+#
+# Original Author: Bryan T. Richardson, Sandia National Laboratories <btricha@sandia.gov>
+#
+# This library is free software; you can redistribute it and/or modify it
+# under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation; either version 2.1 of the License, or (at
+# your option) any later version.
+#
+# This library is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this library; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 
+#
+# This script is modeled after the Rails boot script.
+
+ANTFARM_ROOT = (ENV['ANTFARM_ROOT'] || File.expand_path(File.dirname(__FILE__) + '/..')).dup unless defined? ANTFARM_ROOT
+
+module Antfarm
+  class << self
+    def boot!
+      unless booted?
+        require ANTFARM_ROOT + '/lib/antfarm/initializer'
+        Antfarm::Initializer.run(:setup)
+      end
+    end
+
+    def booted?
+      defined? Antfarm::Initializer
+    end
+  end
+end
+
+Antfarm.boot!

data/config/environment.rb

@@ -0,0 +1,30 @@
+# Copyright (2008) Sandia Corporation.
+# Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
+# the U.S. Government retains certain rights in this software.
+#
+# Original Author: Bryan T. Richardson, Sandia National Laboratories <btricha@sandia.gov>
+#
+# This library is free software; you can redistribute it and/or modify it
+# under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation; either version 2.1 of the License, or (at
+# your option) any later version.
+#
+# This library is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this library; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 
+#
+# This script is modeled after the Rails environment configuration script.
+
+ANTFARM_ENV       = (ENV['ANTFARM_ENV'] || 'antfarm').dup unless defined? ANTFARM_ENV
+ANTFARM_LOG_LEVEL = (ENV['ANTFARM_LOG_LEVEL'] || 'warn').dup unless defined? ANTFARM_LOG_LEVEL
+
+require File.dirname(__FILE__) + '/boot'
+
+Antfarm::Initializer.run do |config|
+  config.log_level = ANTFARM_LOG_LEVEL
+end

data/lib/antfarm-core.rb

@@ -0,0 +1,167 @@
+################################################################################
+#                                                                              #
+# Copyright (2008-2010) Sandia Corporation. Under the terms of Contract        #
+# DE-AC04-94AL85000 with Sandia Corporation, the U.S. Government retains       #
+# certain rights in this software.                                             #
+#                                                                              #
+# Permission is hereby granted, free of charge, to any person obtaining a copy #
+# of this software and associated documentation files (the "Software"), to     #
+# deal in the Software without restriction, including without limitation the   #
+# rights to use, copy, modify, merge, publish, distribute, distribute with     #
+# modifications, sublicense, and/or sell copies of the Software, and to permit #
+# persons to whom the Software is furnished to do so, subject to the following #
+# conditions:                                                                  #
+#                                                                              #
+# The above copyright notice and this permission notice shall be included in   #
+# all copies or substantial portions of the Software.                          #
+#                                                                              #
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR   #
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,     #
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE  #
+# ABOVE COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, #
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR #
+# IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE          #
+# SOFTWARE.                                                                    #
+#                                                                              #
+# Except as contained in this notice, the name(s) of the above copyright       #
+# holders shall not be used in advertising or otherwise to promote the sale,   #
+# use or other dealings in this Software without prior written authorization.  #
+#                                                                              #
+################################################################################
+
+require 'ipaddr'
+
+require File.dirname(__FILE__) + '/../config/boot'
+
+require 'antfarm/helpers'
+require 'antfarm/models'
+require 'antfarm/version'
+
+module Antfarm
+  # Some explanation to having @netmask and such:
+  #   If you create a new IPAddr object and you include
+  #   the network information for the IP address, IPAddr
+  #   doesn't keep track of the actual address, and
+  #   instead just keeps track of the network.  For
+  #   example, if you were to create a new IPAddr object
+  #   using the following code:
+  #
+  #   IPAddr.new("192.168.101.5/24")
+  #
+  #   the resulting object would be of the form:
+  #
+  #   <IPAddr: IPv4:192.168.101.0/255.255.255.0>
+  #
+  #   and there would be no way to retrieve the original
+  #   address (192.168.101.5).  By creating this class,
+  #   Michael has made it possible to keep track of both
+  #   the address and the network information.  This is
+  #   useful in the case of creating a new IPInterface
+  #   object.
+  #
+  # TODO: If a netmask is given, should we somehow check
+  #       to see if an address is being given with network
+  #       information or if a network is being specified,
+  #       and if it is a network, should we validate that
+  #       the network address is valid with the given
+  #       netmask?  This may be done automatically... I
+  #       need to look more into how IPAddr works.
+
+  class IPAddrExt < IPAddr
+    def initialize(value)
+      address,netmask = value.split('/')
+      super(address)
+
+      if self.ipv4?
+        @netmask = IPAddr.new('255.255.255.255')
+        @addr_bits = 32
+      elsif self.ipv6?
+        @netmask = IPAddr.new('ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff')
+        @addr_bits = 128
+      else
+        #TODO: Error
+      end
+   
+      if netmask
+        @netmask = @netmask.mask(netmask)
+      end
+    end
+
+    attr_accessor :netmask
+
+    def netmask_length
+      mask_len = @addr_bits
+      unless (~@netmask).to_i == 0
+        res = Math.log((~@netmask).to_i) / Math.log(2)
+        if res.finite?
+          mask_len -= res.round
+        end
+      end
+      return mask_len
+    end
+
+    def network
+      return self.mask(self.netmask.to_s)
+    end
+
+    def to_cidr_string
+      return sprintf("%s/%s", self.network.to_string, self.netmask_length.to_s)
+    end
+
+    def broadcast
+      return self.network | ~self.netmask
+    end
+   
+    # TODO: track down the IPv6 private use ranges and include them
+    def private_address?
+      private_addr_list = [ '10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16',
+                            'fe80::/10', 'fec0::/10' ]
+      return self.in_address_list?(private_addr_list)
+    end
+
+    #TODO: track down IPv6 localnet mask (guessing /10 for now)
+    def loopback_address?
+      loopback_addr_list = ['127.0.0.0/8', '::1', 'fe00::/10']
+      return self.in_address_list?(loopback_addr_list)
+    end
+
+    # Need to verify the IPv4 multicast addrs (couldn't find the whole
+    # block, only the currently assigned ranges within the block)
+    def multicast_address?
+      multicast_addr_list = ['224.0.0.0/4', 'ff00::/8']
+      return self.in_address_list?(multicast_addr_list)
+    end
+
+    def in_address_list?(addr_str_list)
+      for addr_str in addr_str_list
+        addr = IPAddr.new(addr_str)
+        if addr.include?(self)
+          return true
+        end
+      end
+      return false    
+    end
+
+    # Decides if the given network is a subset of this network.
+    # This method was added since SQLite3 cannot handle CIDR's
+    # 'natively' like PostgreSQL can. Note that this method
+    # also works if the network given is actually a host.
+    def network_in_network?(network)
+      broadcast = nil
+
+      if network.kind_of?(String)
+        broadcast = IPAddrExt.new(network).broadcast
+        network = IPAddr.new(network)
+      elsif network.kind_of?(Antfarm::IPAddrExt)
+        broadcast = network.broadcast
+        network = IPAddr.new(network.to_cidr_string)
+      else
+        raise(ArgumentError, "argument should be either a String or an Antfarm::IPAddrExt object", caller)
+      end
+
+      return false unless IPAddr.new(self.to_cidr_string).include?(network)
+      return false unless IPAddr.new(self.to_cidr_string).include?(broadcast)
+      return true
+    end
+  end
+end

data/lib/antfarm/errors.rb

@@ -0,0 +1,42 @@
+################################################################################
+#                                                                              #
+# Copyright (2008-2010) Sandia Corporation. Under the terms of Contract        #
+# DE-AC04-94AL85000 with Sandia Corporation, the U.S. Government retains       #
+# certain rights in this software.                                             #
+#                                                                              #
+# Permission is hereby granted, free of charge, to any person obtaining a copy #
+# of this software and associated documentation files (the "Software"), to     #
+# deal in the Software without restriction, including without limitation the   #
+# rights to use, copy, modify, merge, publish, distribute, distribute with     #
+# modifications, sublicense, and/or sell copies of the Software, and to permit #
+# persons to whom the Software is furnished to do so, subject to the following #
+# conditions:                                                                  #
+#                                                                              #
+# The above copyright notice and this permission notice shall be included in   #
+# all copies or substantial portions of the Software.                          #
+#                                                                              #
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR   #
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,     #
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE  #
+# ABOVE COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, #
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR #
+# IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE          #
+# SOFTWARE.                                                                    #
+#                                                                              #
+# Except as contained in this notice, the name(s) of the above copyright       #
+# holders shall not be used in advertising or otherwise to promote the sale,   #
+# use or other dealings in this Software without prior written authorization.  #
+#                                                                              #
+################################################################################
+
+module Antfarm
+  class AntfarmError < RuntimeError
+    def initialize(message)
+      super
+
+      message = "#{self.class}: #{message}"
+      Antfarm::Helpers.output("Exception: #{message}")
+      Antfarm::Helpers.log :error, message
+    end
+  end
+end