pcapr-local 0.1.10

data/.document

@@ -0,0 +1,5 @@
+lib/**/*.rb
+bin/*
+- 
+features/**/*.feature
+LICENSE.txt

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Mu Dynamics
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,64 @@
+# pcapr.Local #
+
+## Introduction
+
+pcapr.Local is a tool for browsing and managing a large repository of packet captures (pcaps). After you tell pcapr.Local where your pcaps are located, it will index them automatically and let you navigate your collection in the comfort of your web browser. pcapr.Local builds on and integrates with [Xtractr](http://code.google.com/p/pcapr/wiki/Xtractr) so you can analyze your pcaps in the Xtractr web UI. The Xtractr web UI is hosted on pcapr.net but talks to a local Xtractr instance (managed by pcapr.Local) and your data never leaves your network.
+
+In addition to managing your pcaps, you can use pcapr.Local to leverage your custom wireshark dissectors when creating Scenarios in Mu Studio. PAR files (described below) created by pcapr.Local can be imported into Mu Studio just like a pcap, but Mu Studio will use your wireshark data to guide Scenario creation.
+
+## Dependencies
+
+### CouchDB
+CouchDB needs to be available. Either or local or remote installation will work. On Ubuntu/Debian you can install CouchDB with:
+
+   $ sudo apt-get install couchdb
+
+### Wireshark
+
+You need to have wireshark installed. In particular the command line "tshark" utility should be available. 
+
+### Ruby
+
+Tested with Ruby 1.8.6, 1.8.7, and 1.9.2.
+
+## Supported environments
+
+Linux only. Sorry.
+
+## Running pcapr.Local
+
+1. Install the gem. 
+2. Run the "startpcapr" executable that is installed with the gem:
+
+    $ startpcapr
+
+This will ask you some basic questions, and will record your answers in a config file at ~/.pcapr_local/config that will be used on subsequent invocations. After collecting configuration information, the server process will continue running in the background and you'll get your prompt back. If you like to keep an eye on what's going on you can tail the pcapr.Local log file with:
+
+    $ tail -F ~/pcapr.Local/log/server.log
+
+3. Add pcaps to the pcap directory you configured (default ~/pcapr.Local/pcaps) and wait a short while for them to be noticed and indexed (about a minute). 
+4. Point your browser to http://localhost:8080 (or whatever you configured).
+5. If you want to stop the pcapr.Local server you can do so with:
+
+    $ stoppcapr
+
+## Creating PAR files
+
+A PAR file (Pcap ARchive) is a format that can be imported onto a Mu Studio to create a Scenario. For purposes of Scenario creation, a PAR file is equivalent to the starting pcap with a couple of exceptions:
+
+1.  The PAR file contains wireshark dissection data from your local wireshark installation. This means you get the full benefits of any custom dissectors you may have.
+2.  When you import a PAR you'll bypass the normal flow selection page and go directly to the Scenario editor.
+
+### In the GUI
+
+Select a pcap in the pcapr.Local browser. The page that opens has a link at the bottom that lets you download a PAR file for that pcap.
+
+### On the Command Line
+
+The gem bundles a CLI tool for creating PAR files called 'pcap2par'. Usage is very simple, just provide a path to your pcap:
+ 
+    $  pcap2par my_traffic.pcap
+
+This will create the PAR file called "export.par" in the current directory. You can optionally specify the output file as a second argument:
+
+    $ pcap2par my_traffic.pcap ~/par_files/my_traffic.par 

data/Rakefile

@@ -0,0 +1,57 @@
+require 'rubygems'
+require 'rake'
+
+require 'jeweler'
+Jeweler::Tasks.new do |gem|
+  # gem is a Gem::Specification... see http://docs.rubygems.org/read/chapter/20 for more options
+  gem.name = "pcapr-local"
+  gem.homepage = "http://github.com/pcapr-local/pcapr-local"
+  gem.license = "MIT"
+  gem.summary = %Q{Manage your pcap collection}
+  gem.description = %Q{Index, Browse, and Query your vast pcap collection.}
+  gem.email = "nbaggott@gmail.com"
+  gem.authors = ["Mu Dynamics"]
+  gem.add_dependency "rest-client", ">= 1.6.1"
+  gem.add_dependency "couchrest", "~> 1.0.1"
+  gem.add_dependency "sinatra", "~> 1.1.0"
+  gem.add_dependency "json", ">= 1.4.6"
+  gem.add_dependency "thin", "~> 1.2.7"
+  gem.add_dependency "rack", "~> 1.2.1"
+  gem.add_dependency "rack-contrib", "~> 1.1.0"
+  # Include your dependencies below. Runtime dependencies are required when using your gem,
+  # and development dependencies are only needed for development (ie running rake tasks, tests, etc)
+  #  gem.add_runtime_dependency 'jabber4r', '> 0.1'
+  #  gem.add_development_dependency 'rspec', '> 1.2.3'
+  gem.add_development_dependency "shoulda", ">= 0"
+  gem.add_development_dependency "bundler", "~> 1.0.0"
+  gem.add_development_dependency "jeweler", "~> 1.5.2"
+  gem.add_development_dependency "rcov", ">= 0"
+
+end
+Jeweler::RubygemsDotOrgTasks.new
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+end
+
+require 'rcov/rcovtask'
+Rcov::RcovTask.new(:rcov) do |test|
+  test.libs << 'test'
+  test.pattern = 'test/**/test_*.rb'
+  test.verbose = true
+end
+
+task :default => :test
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  version = File.exist?('VERSION') ? File.read('VERSION') : ""
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "pcapr-local #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end

data/VERSION

@@ -0,0 +1 @@
+0.1.10

data/bin/pcap2par

@@ -0,0 +1,49 @@
+#!/usr/bin/env ruby
+# Copyright (C) 2008 Mu Dynamics, Inc
+#
+# This program is confidential and proprietary to Mu Dynamics, Inc and
+# may not be reproduced, published or disclosed to others without its
+# authorization.
+
+libdir = File.dirname(__FILE__) + "/../lib"
+libdir = File.expand_path(libdir)
+$: << libdir
+
+require 'pcapr_local'
+require 'optparse'
+require 'mu/pcap'
+require 'mu/scenario/pcap'
+
+PcaprLocal::Config.assert_environment
+
+options = {
+    :isolate_l7 => false
+}
+
+opts = OptionParser.new do |opts|
+    opts.banner =
+        "Usage: pcap2par [options] <pcap> [export file]"
+    opts.on('-i', '--isolate', 'Include only TCP/UDP/SCTP traffic (excluding  DNS, DHCP)') do
+        options[:isolate_l7] = true
+    end
+    opts.on_tail('-h', '--help', 'Show this message') do
+        puts opts
+        exit 0
+    end
+end
+
+argv = opts.parse!
+unless argv.size == 1 or argv.size == 2
+    $stderr.puts opts
+    exit 1
+end
+
+pcap    = argv[0]
+archive = argv[1] || "export.par"
+io = Mu::Scenario::Pcap.export_to_par pcap, options
+archive_io = open(archive, 'wb')
+while block=io.read(4096)
+    archive_io.print block
+end
+
+puts "export is located at #{archive}"

data/bin/startpcapr

@@ -0,0 +1,40 @@
+#!/usr/bin/env ruby
+
+libdir = File.dirname(__FILE__) + "/../lib"
+libdir = File.expand_path(libdir)
+$: << libdir
+
+require 'pcapr_local'
+require 'optparse'
+
+PcaprLocal::Config.assert_environment
+
+config_file = nil
+debug_mode = false
+opts = OptionParser.new do |opts|
+    opts.banner = "Usage: #{$0} [-f config_file]"
+    opts.on('-f', '--config_file FILE', 'Config file') do |f|
+        config_file = f
+    end
+    opts.on('-d', '--debug_mode', 'Run in debug mode (server runs in foreground)') do 
+        debug_mode = true
+    end
+    opts.on_tail('-h', '--help', 'Show this message') do
+        puts opts
+        exit 0
+    end
+end
+opts.parse!
+
+config = PcaprLocal::Config.config config_file
+if debug_mode
+    config["debug_mode"] = true
+    # log to stdout
+    config["log_dir"] = nil 
+else
+    config["debug_mode"] = false
+end
+
+
+PcaprLocal.start config
+

data/bin/stoppcapr

@@ -0,0 +1,33 @@
+#!/usr/bin/env ruby
+
+libdir = File.dirname(__FILE__) + "/../lib"
+libdir = File.expand_path(libdir)
+$: << libdir
+
+require 'pcapr_local'
+require 'optparse'
+
+PcaprLocal::Config.assert_environment
+
+include PcaprLocal
+
+config_file = nil
+opts = OptionParser.new do |opts|
+    opts.banner = "Usage: #{$0} [-f config_file]"
+    opts.on('-f', '--config_file FILE', 'Config file') do |f|
+        config_file = f
+    end
+    opts.on_tail('-h', '--help', 'Show this message') do
+        puts opts
+        exit 0
+    end
+end
+opts.parse!
+
+config_file ||= PcaprLocal::Config.user_config_path
+
+if File.exist?(config_file)
+    config = PcaprLocal::Config.config config_file
+    PcaprLocal.stop config
+end
+

data/bin/xtractr

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+xtractr = File.dirname(__FILE__) + "/../lib/exe/xtractr"
+xtractr = File.expand_path xtractr
+exec xtractr, *ARGV

data/lib/environment.rb

@@ -0,0 +1,106 @@
+# http://www.mudynamics.com
+# http://labs.mudynamics.com
+# http://www.pcapr.net
+
+if defined? Encoding
+    Encoding.default_external = Encoding::BINARY
+end
+
+module PcaprLocal
+    ROOT = File.expand_path(File.dirname(File.dirname(__FILE__)))
+    $: << ROOT
+end
+
+class Integer
+    # Make sure Integer#ord is present
+    if RUBY_VERSION < "1.8.7" 
+        def ord
+            return self
+        end
+    end
+end 
+
+# Make sure barebones Dir.mktmpdir is present
+require 'tempfile'
+class Dir
+    if not self.respond_to? :mktmpdir
+        def self.mktmpdir
+            t = (Time.now.to_f * 1_000_000).to_i.to_s(36)
+            path = "#{tmpdir}/d#{t}-#{$$}-#{rand(0x100000000).to_s(36)}"
+            Dir.mkdir path
+            path
+        end 
+    end
+end
+
+
+module Process
+    #  Supply daemon for pre ruby 1.9
+    #  Adapted from lib/active_support/core_ext/process/daemon.rb
+    def self.daemon(nochdir = nil, noclose = nil)
+        exit! if fork                     # Parent exits, child continues.
+        Process.setsid                    # Become session leader.
+        exit! if fork                     # Zap session leader. See [1].
+
+        unless nochdir
+            Dir.chdir "/"                 # Release old working directory.
+        end
+
+        unless noclose
+            STDIN.reopen "/dev/null"       # Free file descriptors and
+            STDOUT.reopen "/dev/null", "a" # point them somewhere sensible.
+            STDERR.reopen '/dev/null', 'a'
+        end
+
+        trap("TERM") { exit }
+
+        return 0
+
+    end unless self.respond_to? :daemon
+end
+
+class Regexp
+    # Patch Regexp.union to accept an array
+    if RUBY_VERSION < "1.8.7" 
+        class << self
+            alias :union_pre187 :union
+            def union *arg
+                if arg.size == 1 and arg[0].is_a? Array
+                    arg = arg[0]
+                end
+                union_pre187 *arg
+            end
+        end
+    end
+end 
+
+class String
+    # Convert from hex.  E.g. "0d0a".from_hex is "\r\n".
+    # Raises ArgumentError on invalid input.
+    def from_hex
+        return "" if self.empty?
+        hex = self
+        Integer("0x#{hex}")
+        if hex.length % 2 == 1
+            hex = "0#{hex}"
+        end
+        [hex].pack 'H*'
+    end
+end
+
+# Implement simple Readline.readline if interpreter is not
+# compiled with readline support.
+begin
+    require 'readline'
+rescue LoadError
+    class Readline
+        def self.readline prompt
+            print prompt
+            gets
+        end
+    end
+end
+
+
+
+

data/lib/mu/pcap.rb

@@ -0,0 +1,110 @@
+# http://www.mudynamics.com
+# http://labs.mudynamics.com
+# http://www.pcapr.net
+
+require 'socket'
+require 'stringio'
+
+module Mu
+
+class Pcap
+    class ParseError < StandardError ; end
+
+    LITTLE_ENDIAN = 0xd4c3b2a1
+    BIG_ENDIAN    = 0xa1b2c3d4
+
+    DLT_NULL      = 0
+    DLT_EN10MB    = 1
+    DLT_RAW       = 12 # DLT_LOOP in OpenBSD
+    DLT_LINUX_SLL = 113
+
+    attr_accessor :header, :pkthdrs
+    
+    def initialize
+        @header = Header.new
+        @pkthdrs = []
+    end
+
+    # Read PCAP file from IO and return Mu::Pcap.  If decode is true, also
+    # decode the Pkthdr packet contents to Mu::Pcap objects.
+    def self.read io, decode=true
+        pcap = Pcap.new
+        pcap.header = each_pkthdr(io, decode) do |pkthdr|
+            pcap.pkthdrs << pkthdr
+        end
+        return pcap
+    end
+
+    # Create PCAP from list of packets.
+    def self.from_packets packets
+        pcap = Pcap.new
+        packets.each do |packet|
+            pkthdr = Mu::Pcap::Pkthdr.new
+            pkthdr.pkt = packet
+            pcap.pkthdrs << pkthdr
+        end
+        return pcap
+    end
+
+    # Write PCAP file to IO.  Uses big-endian and linktype EN10MB.
+    def write io
+        @header.write io
+        @pkthdrs.each do |pkthdr|
+            pkthdr.write io
+        end
+    end
+
+    # Read PCAP packet headers from IO and return Mu::Pcap::Header.  If decode
+    # is true, also decode the Pkthdr packet contents to Mu::Pcap objects.  Use
+    # this for large files when each packet header can processed independently
+    # - it will perform better.
+    def self.each_pkthdr io, decode=true
+        header = Header.read io
+        while not io.eof?
+            pkthdr = Pkthdr.read io, header.magic
+            if decode
+                pkthdr.decode! header.magic, header.linktype
+            end
+            yield pkthdr
+        end
+        return header
+    end
+
+    # Read packets from PCAP
+    def self.read_packets io, decode=true
+        packets = []
+        each_pkthdr(io) { |pkthdr| packets << pkthdr.pkt }
+        return packets
+    end
+
+    # Assertion used during Pcap parsing
+    def self.assert cond, msg
+        if not cond
+            raise ParseError, msg
+        end
+    end
+
+    # Warnings from Pcap parsing are printed using this method.
+    def self.warning msg
+        $stderr.puts "WARNING: #{msg}"
+    end
+
+    def == other
+        return self.class == other.class &&
+            self.header   == other.header &&
+            self.pkthdrs  == other.pkthdrs
+    end
+end
+
+end
+
+require 'mu/pcap/header'
+require 'mu/pcap/pkthdr'
+require 'mu/pcap/packet'
+require 'mu/pcap/ethernet'
+require 'mu/pcap/ip'
+require 'mu/pcap/ipv4'
+require 'mu/pcap/ipv6'
+require 'mu/pcap/tcp'
+require 'mu/pcap/udp'
+require 'mu/pcap/sctp'