tlspretense 0.6.1

data/.document

@@ -0,0 +1,6 @@
+lib/**/*.rb
+bin/*
+- 
+features/**/*.feature
+LICENSE.txt
+

data/.gitignore

@@ -0,0 +1,7 @@
+certs
+coverage
+rdoc
+.yardoc
+.bundle
+pkg
+tags

data/.rspec

@@ -0,0 +1 @@
+--color

data/Gemfile

@@ -0,0 +1,2 @@
+source "http://rubygems.org"
+gemspec

data/Gemfile.lock

@@ -0,0 +1,41 @@
+PATH
+  remote: .
+  specs:
+    tlspretense (0.6.0)
+      eventmachine (>= 1.0.0)
+      ruby-termios (>= 0.9.6)
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    diff-lcs (1.1.3)
+    eventmachine (1.0.0)
+    json (1.7.5)
+    multi_json (1.5.0)
+    rake (10.0.3)
+    rdoc (3.12)
+      json (~> 1.4)
+    rspec (2.8.0)
+      rspec-core (~> 2.8.0)
+      rspec-expectations (~> 2.8.0)
+      rspec-mocks (~> 2.8.0)
+    rspec-core (2.8.0)
+    rspec-expectations (2.8.0)
+      diff-lcs (~> 1.1.2)
+    rspec-mocks (2.8.0)
+    ruby-termios (0.9.6)
+    simplecov (0.7.1)
+      multi_json (~> 1.0)
+      simplecov-html (~> 0.7.1)
+    simplecov-html (0.7.1)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  bundler (~> 1.0)
+  rake (>= 0.8.7)
+  rdoc (~> 3.12)
+  rspec (~> 2.8.0)
+  simplecov (~> 0.7)
+  tlspretense!

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2012 iSEC Partners
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,231 @@
+= TLSPretense --- SSL/TLS Client Testing Framework
+
+A test framework for testing SSL/TLS client certificate validation.
+
+== Description
+
+Note: TLSPretense is still undergoing a lot of polishing. It is currently
+usable, but features may change and documentation may be missing. As such,
+please bear with us over next few months as we find time to work on the tools,
+and feel free to file a bug with details.
+
+TLSPretense provides a test framework for testing SSL certificate validation.
+It generates a set of certificates containing specific flaws, and it presents
+the certificates to a client that has been configured to trust a CA used by
+TLSPretense. The test framework then configures its system's firewall to
+redirect and intercept network traffic so that the test runner can present its
+certificate to the client. To speed up testing, the test runner starts the next
+test as soon as the current test finishes.
+
+The test framework must be run on a Unix-like OS that contains a supported
+firewall, but the program being tested can run on any device whose network
+traffic can be routed through the system hosting the test framework. Currently,
+it supports netfilter (Linux), ipfw (Mac OS X 10.6, *BSD), and PF on Mac OS X
+Lion.
+
+It also has an implementation for a newer version of PF, although this is
+untested.
+
+
+== Links
+
+* {Generated Documentation}[http://isecpartners.github.com/tlspretense/]
+
+
+== How It Works
+
+TLSPretense requires the TLS client software to be configured to trust a CA
+that TLPretense controls. That way "good" certificates created by TLSPretense
+will be accepted by the client.
+
+Once the system hosting the test runner has been configured to be a gateway for
+the network traffic of thest client being tested, it will add a firewall rule
+to redirect network traffic to a test listener. The test listener checks to see
+whether the client is trying to connect to a predefined host. If the client is
+connecting to the desired host, then the test listener presents a test
+certificate chain to the client. The test runner then determines whether the
+test passes or fails based on whether the client completes the TLS handshake or
+not.
+
+The test harness was designed to anticipate working with a client that may
+connect to more than one host. The config.yml file specifies a hostname that
+should be used for the actual test --- all other intercepted SSL connections
+are essentially ignored (although they currently have their certificate
+re-signed by the goodca in order to make interception easier).
+
+== Requirements
+
+* A Unix-like system that uses a supported firewall/routing implementation.
+  TLSPretense currently supports:
+  * Netfilter on Linux
+  * IPFW on MacOSX 10.6 and earlier, and *BSD
+  * PFRdr on MacOSX 10.7 (and probably also 10.8)
+
+* Ruby 1.9.x (Developed with 1.9.3). Check your version with:
+
+      ruby --version
+
+  Some systems will install Ruby 1.9.x with a suffix, like `ruby1.9`. Ruby must
+  also be built against a version of OpenSSL that supports the SNI TLS
+  extension. You can check for this if you run the following Ruby script (on
+  some systems, Ruby 1.9.x will be installed as ruby1.9, and commands like gem
+  will also have the 1.9 suffix):
+
+      ruby -ropenssl -e 'puts OpenSSL::SSL::SSLSocket.public_instance_methods.include? :hostname='
+
+  Ruby 1.8.7 will mostly work, but Ruby 1.8's OpenSSL wrapper library does not
+  support the ability for clients to use the SNI TLS extension, which is needed
+  to grab the correct remote certificate for proxying miscellaneous
+  connections. Use Ruby 1.8.x at your own risk.
+
+* The SSL client/HTTPS user agent has to trust the CA used by TLSPretense.
+  You can either generate a new goodca and install it in the client's trust
+  store, or you can use an existing test CA with TLSPretense to generate the
+  test certificates.
+
+== Quick Start
+
+Install with rubygems:
+
+    umask 0022 ; sudo gem install tlspretense
+
+Create a new project:
+
+    tlspretense init myproject
+    cd myproject
+
+And edit config.yml to suit your needs. If you want to create a new test CA
+(not necessary if you want to use the default or your own):
+
+    tlspretense ca
+
+Generate certificates for the test cases:
+
+    tlspretense certs
+
+You will also need to setup the host's networking stack and firewall to support
+TLSPretense. More details can be found in {TLSPretense
+Setup}[rdoc-ref:general_setup] and in the system-specific guides.
+
+Finally, run all of the configured test cases:
+
+    sudo tlspretense run
+
+Or just certain tests (in the order specified):
+
+    sudo tlspretense run unknownca wrongcname
+
+== Limitations
+
+* The Server Name Indication (SNI) TLS extension does not have full support in
+  Ruby 1.8.7.
+
+* Protocols that explicitly call STARTTLS to enable SSL/TLS (eg, SMTP and IMAP)
+  are not yet supported. They would require protocol-specific support. The
+  version of these protocols where they are wrapped in SSL should be testable
+  though.
+
+* It currently uses the goodca to re-sign certificates from hostnames that do
+  not match the configured test hostname.
+
+* The existing PFDivert rule implementation does not work on Mac OS X 10.7 (and
+  probably not on 10.8 either). OpenBSD 5.0(?) and FreeBSD 9 can make use of
+  this functionality though.
+
+== TODO
+
+* Convert SSLClient's initial connection to use a non-blocking connect.
+
+* Change the pre-flight in SSLSmartProxy to disable the accepted server socket
+  until the pre-flight finishes. (the SSLClient within SSLTransparentProxy
+  probably also should do this until the client connects)
+
+* Fix the subjectAltName null byte test. It requires manual construction of the
+  ASN1 encoding due to the null byte causing a truncation somewhere in OpenSSL
+  or Ruby's OpenSSL.
+
+* Add wildcard tests
+  * cert hostname: \*.%PARENTHOSTNAME%
+  * bad hostname: \*.other.com
+  * tld hostname: \*.%TLDHOSTNAME% (reject)
+  * do we want to test more complicated wildcards?
+
+* Decide how to deal with a wildcard cert at the original destination. If we
+  are testing foo.somehost.com, and the client connects to other hostnames like
+  bar.somehost.com, and they all use a *.somehost.com certificate, then
+  TLSPretense gets confused.
+
+* Add extension criticality tests (need an extension oid that nobody knows how to verify)
+  * If an extension is marked critical and the verifier doesn't know how to
+    verify, then it should fail. If it is not marked critical, then it is
+    allowed to skip the extension.
+
+* Advanced: Add name constraints tests.
+  * success: dnsName of leaf matches exactly the dnsName permitted constraint
+    nameConstraints=permitted;dnsName:%HOSTNAME%
+  * reject: constraint is a different hostname
+    nameConstraints=permitted;dnsName:some.other.com
+  * success: dnsName of leaf is a subdomain in addition to dnsName constraint
+    constraint = parent domain of hostname (need to ensure hostname has enough labels)
+    nameConstraints=permitted;dnsName:%PARENTHOSTNAME%
+    do it this way vs trying a subdomain of the original hostname to
+  * reject: constraint is a slightly different hostname
+    nameConstraints=permitted;dnsName:a%HOSTNAME%
+  * success: dirname matches the default subject's DN
+  * reject: dirname does not match the default subject's DN
+  * URI constraints
+
+* Document how to run SSLTest from MacOSX
+
+* Document how to run SSLTest from a Linux VM on Windows (eg, with VMWare)
+
+* Document how to deal with certificate pinning and other things that may make
+  testing certificate validation logic difficult.
+
+* Command line option to specify where to write the results to
+
+* Add more result output formats
+  * (X)HTML
+  * CSV
+  * LaTeX?
+  * XML?
+  * SQLite?
+
+* truly unique serial numbers for a given CA. Alternatively, we could use the
+  first cert's serial as a starting point and increment from there.
+
+* Build certs and chains of certs for each test so that something like
+  s\_server could use them.
+
+* Make initialization interactive. It should prompt the user to choose or
+  confirm configuration details like the interception/firewalling method to
+  use, the network device to listen on, the default hostname, etc. It could
+  then auto-generate the necessary certificates as well.
+
+* Config file validator
+
+* Add an API for interacting with an external test controller. This could be a
+  little web service, although that lacks real-time responses. A TCP/unix
+  socket interface that sends/receives JSON messages (or something simpler)
+  might be better. The client would tell TLSPretense which test to start, and
+  then TLSPretense would reply with a result when it completes.
+
+== Contributing to TLSPretense
+
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
+* Fork the project.
+* Start a feature/bugfix branch.
+* Commit and push until you are happy with your contribution.
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+== Authors
+
+* William (B.J.) Snow Orvis (iSEC Partners)
+
+== Copyright
+
+Copyright (c) 2012 iSEC Partners
+
+See LICENSE.txt for further details.

data/Rakefile

@@ -0,0 +1,44 @@
+
+require 'bundler/gem_tasks'
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec) do |spec|
+  spec.pattern = FileList['spec/**/*_spec.rb']
+end
+
+namespace(:spec) do
+  desc "Create rspec code coverage"
+  task :coverage do
+    ENV['COVERAGE'] = 'true'
+    Rake::Task["spec"].execute
+  end
+end
+
+task :default => :spec
+
+require 'rdoc/task'
+Rake::RDocTask.new do |rdoc|
+  require 'tlspretense/version'
+  version = TLSPretense::VERSION
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.main = 'README.rdoc'
+  rdoc.title = "TLSPretense #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+  rdoc.rdoc_files.include('doc/**/*.rdoc')
+
+  rdoc.before_running_rdoc do
+    unless File.exist? 'rdoc'
+      begin
+        sh 'git clone --branch gh-pages --single-branch `git config remote.origin.url` rdoc'
+      rescue
+      end
+    end
+  end
+end
+
+require 'certmaker/tasks'
+
+desc "Runs certs:clean"
+task :clean => ['certs:clean']

data/bin/makeder.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+
+# Output in a more compatible format (Opera rejects PEM...)
+echo "Creating a DER version of the cert:"
+echo "==================================="
+openssl x509 -in $1 -inform PEM -out ${1%.pem}.der -outform DER

data/bin/tlspretense

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+
+$:.unshift(File.dirname(__FILE__)+ '/../lib') unless $:.include?(File.dirname(__FILE__)+ '/../lib')
+
+require 'tlspretense'
+
+TLSPretense::App.new(ARGV,$stdin,$stdout).run

data/bin/view.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+
+openssl x509 -in $1 -noout -text

data/doc/general_setup.rdoc

@@ -0,0 +1,288 @@
+= TLSPretense Setup
+
+The following sections cover a basic configuration to get TLSPretense up and
+running. It should be straightforward to modify the test setup from there.
+
+== Test Environment
+
+TLSPretense was designed to be run as an intercepting proxy in order to be able
+to test a client that may make multple HTTPS/SSL/TLS requests to multiple
+remote hosts. This means that TLSPretense is expected to run on a different
+system from the client software being tested, although it is possible to run
+TLSPretense within one virtual machine and to have the client software run in
+another.
+
+First, a few definitions:
+
+[client]            A piece of software that makes network requests using
+                    SSL/TLS.
+[client host]       The computer or virtual machine that will run the client
+                    software.
+[TLSPretense host]  A computer or virtual machines running a Unix-like OS that
+                    will act as a network gateway for the client host.
+
+The TLSPretense host needs to have two network interfaces: an external
+interface for connecting to the outer LAN/Internet and to the remote hosts that
+the client will connect to, and an internal interface that the client host will
+use as its gateway.
+
+One common setup is to use a laptop for the TLSPretense host with an ethernet
+connection and wifi. The ethernet can be wired into an actual network, and the
+wifi can be run as a wireless access point that the client host will connect
+to.
+
+This setup has the following network topography:
+
+    +------------------------+  LAN   +------------------+  WAN
+    |      Client Host       |   .    | TLSPretense Host |   .
+    | +--------------------+ |   .    |                  |   .
+    | |       Client       | |   .    |   +----------+   |   .
+    | |                    ==============>| Firewall |   |   .
+    | | +----------------+ | |   .    |   +----------+   |   .
+    | | | CA TrustStore  | | |   .    |        |         |   .    +-------------+
+    | | |                | | |   .    | +------V-------+ |   .    | Original    |
+    | | | TLSPretense CA | | |   .    | | TLSPretense  ==========>| Destination |
+    | | +----------------+ | |   .    | | Test Harness | |   .    +-------------+
+    | +--------------------+ |   .    | +--------------+ |   .    
+    +------------------------+   .    +------------------+   .
+
+
+== Installing TLSPretense
+
+Install with rubygems:
+
+    umask 0022 ; sudo gem install tlspretense
+
+Create a new project:
+
+    tlspretense init myproject
+    cd myproject
+
+Now edit the config.yml file to suit your needs. TLSPretense installs a default
+CA when you create your project directory, but you can replace it with your own
+test CA. You can also create a new CA if you want (using the goodca certificate
+definition in the project's config.yml):
+
+    tlspretense ca
+
+Next, create certificates for the test cases (you should rerun this if you
+modify any of the certificate descriptions in config.yml):
+
+    tlspretense certs
+
+You will also need to setup the host's networking stack and firewall to support
+TLSPretense. More details can be found in {TLSPretense
+Setup}[rdoc-ref:general_setup] and in the system-specific guides.
+
+Finally, to run all of the configured test cases:
+
+    sudo tlspretense run
+
+Or just certain tests (in the order specified):
+
+    sudo tlspretense run unknownca wrongcname
+
+== Updating TLSPretense
+
+Just install TLSPretense again using rubygems. At present, TLSPretense does not
+provide any versioning or migration of its configuration file.
+
+== Configuration
+
+All configuration of the test suite happens through your project's config.yml.
+What follows is a brief discussion of each section and what you might want to
+change.
+
+=== top-level settings
+
+[hostname: www.isecpartners.com]
+    This is the most important setting in the configuration file. It specifies
+    the hostname that the client will connect to that you would like to have
+    tested. For connections to this hostname (determined by whether the
+    certificate supplied by the original destination matches the hostname),
+    TLSPretense will present a test certificate chain. For all other hosts, it
+    will take the host certificate used by that host and re-sign it using the
+    "goodca" certificate.
+
+[listener_port: 54321]
+    The port that the proxy will listen on. You probably don't need to change
+    this.
+
+=== packetthief
+
+    packetthief:
+      protocol: tcp
+      dest_port: 443
+      in_interface: eth1
+
+This section describes the firewall rule that will redirect traffic. The most
+relevant subvalue is +:in_interface+, which should contain the name of the
+network interface that the client's network traffic will come in on. Also, if
+you will be testing non-https traffic, change +dest_port+ to the port that the
+client will connect to.
+
+One additional optional parameter exists in this this section:
+
+[implementation]
+    If this optional value is set, then the specified implementation will be
+    used. If implementation is left commented out, then TLSPretense will choose
+    the most appropriate firewall implementation, if possible. Current values:
+
+    * *netfilter* Linux netfilter
+    * *ipfw* MacOSX/BSD ipfw
+
+=== certmaker
+
+    certmaker:
+      defaultsubject: &defaultsubject "C=US, CN=%HOSTNAME%"
+      defaultsubjectwithnull: &defaultsubjectwithnull "C=US, CN=%HOSTNAME%\0.foo.com"
+      defaultparentsubject: &defaultparentsubject "C=US, CN=%PARENTHOSTNAME%"
+      outdir: certs
+      missing_serial_generation: random
+      customgoodca:
+        certfile: ca/goodcacert.pem
+        keyfile:  ca/goodcakey.pem
+        #keypass: changeme
+
+This section deals with configuring the creation of the certificates used by
+the tests.
+
+[+defaultsubject+, +defaultsubjectwithnull+, and +defaultparentsubject+]
+    These keys are just placed here to get them out of the way. They are
+    referenced later in the certificate definitions.
+[outdir]
+    This specifies where to write the generated certificates to. For now, don't
+    modify this option or it may break something.
+[missing_serial_generation]
+    This key specifies how to choose a serial number for each generated
+    certificate. Currently, there are only two options. You can set a number,
+    which will be used as the serial number for all generated certificates
+    (this will break any SSL client that checks for serial number uniqueness).
+    The other option is to set it to "random" which will generate a random
+    serial number --- this is the recommended setting, and there is an
+    incredibly small chance that two certificates might have the same serial
+    number.
+[customgoodca]
+    If this key exists and contains the +certfile+ and +keyfile+ sub-keys, then
+    it specifies a certificate authority to be used as the "goodca" instead of
+    generating a new CA. The third optional value, +keypass+, can specify a
+    password for the private key's file.
+
+=== logger
+
+    logger:
+      level: INFO
+      file: '-'
+
+These options set up the logging facility.
+
+[level]
+    The log severity to report. Log messages that are less severe than this
+    level go unreported.
+[file]
+    Where to write log entries to. The default of '-' refers to STDOUT.
+
+=== certs
+
+This section defines the certificates used by the test cases. It makes heavy
+use of YAML's references and anchors to mutate a particular certificate. The
+only special entry is "goodca". If +customgoodca+ is configured in the
++certmaker+ section, then that certificate will be used in tests in place of
+the goodca defined here.
+
+Make changes to the certificates if you need to customize them for your client.
+
+=== tests
+    tests:
+    - alias: baseline
+      name: Baseline Happy Test
+      certchain:
+      - baseline
+      - goodca
+      expected_result: connected
+    ...
+
+
+This final section defines a list of all of the tests to run.
+
+[alias]
+    The "short" identifier for the test.
+[name]
+    A longer description of the test.
+[certchain]
+    A list of certificates to present to the client in the order they should be
+    presented. SSL/TLS requires the certificate chain to present the
+    certificate representing the server first, followed by the chain of
+    certificate authorities and intermediate CAs, with the final certificate
+    being a CA.
+[expected_result]
+    The expected outcome for the test. "connected" means that we expect the
+    client to finish the TLS handshake, while "rejected" means we expect the
+    TLS handshake to fail before finishing.
+
+
+== Certificate Generation
+
+You need to pre-generate the certificates you'll use for the tests. You can do
+this from your project directory with:
+
+    tlspretense certs
+
+It can generate an entire ecosystem of certificates from scratch, or it can use
+an existing CA instead of generating a new _goodca_. See the _certmaker_ and
+_certs_ sections of config.yml.
+
+WARNING: Running this command a second time will regenerate all of the
+certificates in the _certs_ directory! However, by default it will just copy
+the CA from the ca directory each time it is run.
+
+It you want to create a new CA, run:
+
+    tlspretense ca
+
+
+== Client Host Setup
+
+The client host must be configured to use the TLSPretense host as its gateway.
+This involves configuring the TLSPretense host to act like a NAT router or
+network bridge, and it requires you to configure the client host. You must also
+install the "goodca" CA used by TLSPretense onto the client host and configure
+the client to trust that CA. You do not need to configure any client-side proxy
+settings (like you would for HTTP-level proxying).
+
+If you are planning to use Linux for the TLSPretense host and you are familiar
+with {Mallory}[http://intrepidusgroup.com/insight/mallory/], then Mallory's
+network configuration tutorials may help you out, although TLSPretense manages
+the firewall rule that redirects network traffic itself.
+
+For a sample Linux-specific configuration, see: {Linux
+Setup}[rdoc-ref:linux_setup].
+
+
+== Running the tests
+
+While in your project directory, run all tests with:
+
+    sudo tlspretense run
+
+The command will more than likely need to be run as root so that it can add and
+remove its firewall rules.
+
+If you only want to run certain tests, you can do so by specifying which tests
+you want to run as command line arguments, and it will run just those tests in
+the order specified, instead of in the order they are defined in config.yml:
+
+    sudo tlspretense run baseline nullincname selfsigned
+
+For information about other command line options run TLSPretense without any
+options, or call the run sub-command with -h or --help:
+
+    tlspretense
+
+    tlspretense run -h
+
+
+== Output
+
+At present, TLSPretense prints a summary of the tests run and their results
+once all tests have run.