hardy 0.0.1

data/.gitignore

@@ -0,0 +1,21 @@
+*.env
+*.siege
+data/
+
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/CHANGELOG.md

@@ -0,0 +1,12 @@
+# hardy changelog
+
+## [HEAD / unreleased][head.diff]
+
+No significant changes.
+
+## 0.0.1 / 2013-02-14
+
+* Initial release.
+
+
+[head.diff]: https://github.com/nbibler/hardy/compare/v0.0.1...master

data/Gemfile

@@ -0,0 +1,2 @@
+source 'https://rubygems.org'
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Nathaniel Bibler
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,173 @@
+# Hardy
+
+Hardy is a [Thor][thor] library which easily converts an [HTTP Archive
+(HAR)][har] file into a [siege][siege] URLs file.
+
+What does that mean for you? Well, it means that it's now a trivial task to
+generate load testing scripts for your HTTP(S) web applications and determine
+exactly how many concurrent, active, hammering-away-on-your-systems users your
+application and infrastructure can fully support.
+
+Stop guessing and find out! It's easy!!
+
+## Installation
+
+Add this line to your application's Gemfile (probably not in the default group,
+but more likely in a :test or :development group):
+
+    gem 'hardy'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install hardy
+
+### Generating HAR files
+
+Creating a HAR file is simple:
+
+1. Open Chrome,
+2. Open the Chrome Developer Tools panel (cmd-shift-i on a Mac),
+3. With the tools panel open, navigate to the site you want to test,
+4. Click the Preserve Log upon Navigation button on the Network tab of the
+   tools panel (otherwise, your Network tab will be cleared with each page
+   view),
+5. Click around the site! Act like the user you want to simulate. Meaning,
+   click links, post forms, change pages, sign in, sign out. Whatever you do
+   here will be recreated by your load test users, so try to do things that
+   will generate interesting test metrics.
+6. Right-click in the Network panel and click "Save as HAR with Content"
+
+You now have a HAR file to source for load testing! :beer:
+
+### Installing siege
+
+Note: If you are planning on testing a site which uses a JSON interface, you'll
+want to read the [Enabling JSON support in
+siege](#enabling-json-support-in-siege) section further down this README.
+
+On a Mac, you can install siege with homebrew:
+
+    $ brew install siege
+
+On Ubuntu:
+
+    $ apt-get install siege
+
+Installing from source:
+
+    $ curl http://www.joedog.org/pub/siege/siege-latest.tar.gz -o siege-latest.tar.gz
+    $ tar xvfz siege-latest.tar.gz
+    $ cd siege-*
+    $ ./configure
+    $ make
+    $ make install
+
+It is important to note that if you want to load test an HTTPS (SSL) site,
+you'll need to install siege with SSL support. The Mac homebrew installation
+and Ubuntu package already enable this, by default.
+
+    $ ./configure --with-ssl
+
+### Enabling JSON support in siege
+
+So, you're load testing a web application which uses JSON, huh? Does it look
+like your application is not recognizing the parameters as JSON? Yeah... as of
+siege 2.74 (currently the latest version), siege does not recognize `.json`
+files, nor does it support automatically setting the `Content-Type:
+application/json` request header. Bummer.
+
+But you need JSON support, you say? Yeah, me too. Sadly, that means you get to
+edit some C source code. So, download the latest source (as instructed in the
+[Installing siege](#installing-siege) section, above) and before you
+`./configure`, you'll need to edit `src/load.c`:
+
+```diff
+static const struct ContentType tmap[] = {
+  {"default", TRUE,  "application/x-www-form-urlencoded"},
+  {"ai",      FALSE, "application/postscript"},
+  ...
++ {"json",    FALSE, "application/json"},
+  ...
+  {"xyz",     FALSE, "chemical/x-pdb"},
+  {"zip",     FALSE, "application/zip"}
+};
+```
+
+Just add an entry to allow `.json` files to be recognized and transmitted as
+"application/json" format. Once that's done, re-configure, build, and install
+siege, as [detailed above](#installing-siege). Now, when the URLs file defines
+a `.json` file, siege will automatically recognize it and make the request with
+the proper `Content-Type` request header.
+
+## Usage
+
+Once you've created or otherwise acquired your HAR file, use Hardy to convert
+it to a [siege URLs file][urls-file]:
+
+    $ hardy convert my-har-file.env
+
+By default, this will create a `urls.siege` file and a `data` directory full of
+data files to support it.
+
+### Hardy options
+
+There are a handful of command line options when running the convert task,
+ranging from defining your output file to changing the data directory, and
+more. Check out `hardy help convert` for more details. Below are a few details
+of the more interesting ones:
+
+`--host-filter` will restrict the generated URLs file to only include requests to
+the given host. This is very useful if your site uses a CDN for assets and you
+do not want to include those requests in your load test (that would be a bad
+idea!).
+
+`--host` will replace the request hosts for all generated URLs with the host
+given. This is useful if you're generating your HAR script on one system (say,
+your staging server) and want to run the siege test against another (perhaps
+your production stack).
+
+`--protocol` will replace the request protocols for all generated URLs with the
+protocol given. This is useful for creating one script where you're load
+testing `https://` and another where you're load testing `http://`. Again,
+commonly useful when you're generating your HAR from a system different from
+the on you're running your siege against.
+
+An example where we've generated a `my-site.env` HAR file by walking our
+staging site (http://staging.my-site.com) and we want to have siege run against
+our production site (https://www.my-site.com):
+
+```
+$ hardy my-site.env --host-filter=staging.my-site.com --host=www.my-site.com --protocol=https://
+```
+
+### Using siege
+
+Now, to use siege with your shiny new URLs file (and data directory), here's an
+example:
+
+    $ siege -c5 -d10 -t5M -v -f urls.siege
+
+This tells siege to run with 5 concurrent users (-c5), ramping them up over 10
+seconds (-d10), running with all 5 users active for 5 minutes (-t5M),
+displaying the request results to the screen (-v), and sourcing your URLs file
+for their run script (-f urls.siege).
+
+siege offers a lot of settings options, so check out `siege --help` for more
+information.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+[har]: http://www.softwareishard.com/blog/har-12-spec/
+[siege]: http://www.joedog.org/siege-home/
+[thor]: https://github.com/wycats/thor
+[urls-file]: http://www.joedog.org/siege-manual/#a05

data/Rakefile

@@ -0,0 +1 @@
+require "bundler/gem_tasks"

data/bin/hardy

@@ -0,0 +1,156 @@
+#!/usr/bin/env ruby
+
+require 'delegate'
+require 'digest'
+
+require 'addressable/uri'
+require 'mime/types'
+require 'har'
+require 'thor'
+
+class RequestWithURI < BasicObject
+  def initialize(request)
+    @request = request
+  end
+
+  def uri
+    @uri ||= ::Addressable::URI.parse(url)
+  end
+
+  def method_missing(method, *args, &block)
+    @request.public_send(method, *args, &block)
+  end
+end
+
+module Hardy
+  class Siege < Thor
+    desc "convert HAR_PATH", "convert a HAR file to a siege URLs file"
+    method_option :data_path, :type => :string, :aliases => '-d', :default => 'data', :desc => 'directory to output request data for POST/PUT requests'
+    method_option :host_filter, :type => :string, :desc => 'filter the HAR content to only a matching host'
+    method_option :force, :type => :boolean, :aliases => '-f', :default => false, :desc => 'override output if it already exists'
+    method_option :host, :type => :string, :aliases => '-h', :desc => 'convert source HAR hosts into a different URL host'
+    method_option :output, :type => :string, :aliases => '-o', :default => 'urls.siege', :desc => 'file path populate with URL content'
+    method_option :protocol, :type => :string, :aliases => '-p', :desc => 'convert source HAR protocol into a different protocol'
+    def convert(har_path)
+      if File.exist?(options[:output])
+        if File.directory?(options[:output])
+          abort("Must specify a filename, #{options[:output].inspect} is a directory")
+        elsif options[:force]
+          FileUtils.rm(options[:output])
+        else
+          abort("Output file already exists, use -f to force.")
+        end
+      end
+
+      siege_lines = []
+      HAR::Archive.from_file(har_path).entries.each do |entry|
+        siege_lines << create_siege_entry_from(RequestWithURI.new(entry.request))
+      end
+
+      File.open(options[:output], 'w') do |file|
+        file.puts siege_lines.compact.join("\n")
+      end
+    end
+
+
+    private
+
+
+    def create_data_file(filename, data)
+      File.open(filename, 'w') do |file|
+        file.write data
+      end
+    end
+
+    def create_siege_entry_from(request)
+      if host_filter && request.uri.host.downcase != host_filter.downcase
+        return
+      end
+
+      siege_url_for(request)
+    end
+
+    def data_filename_for(request)
+      post_data = request.post_data.text rescue nil
+
+      if post_data
+        content_type = request.headers.detect { |header| header['name'] == 'Content-Type' } || {'value' => 'text/html'}
+        content_type = content_type['value'].split(';', 2).first
+        mime_type = MIME::Types[content_type].first
+
+        "%{data_path}%{basename}.%{extension}" % {
+          data_path: data_path,
+          basename: Digest::MD5.hexdigest(request.post_data.text),
+          extension: mime_type ? mime_type.extensions.first : 'postdata'
+        }
+      end
+    end
+
+    def data_path
+      if options[:data_path]
+        FileUtils.mkdir(options[:data_path]) unless File.exist?(options[:data_path])
+        options[:data_path] + "/"
+      end
+    end
+
+    def host_filter
+      options[:host_filter]
+    end
+
+    def host_for(uri)
+      options[:host] || uri.host
+    end
+
+    def protocol_for(uri)
+      if protocol = options[:protocol]
+        options[:protocol] =~ /:\/\// ?
+          options[:protocol] :
+          options[:protocol] + "://"
+      else
+        "%s://" % uri.scheme
+      end
+    end
+
+    def host_for(uri)
+      options[:host] || uri.host
+    end
+
+    def siege_url_for(request)
+      url = url_for(request.uri)
+
+      case request.method
+      when 'GET'
+        url
+      when 'POST', 'PUT'
+        if filename = data_filename_for(request)
+          create_data_file(filename, request.post_data.text)
+          "%{url} %{method} < %{filename}" % {
+            url: url,
+            method: request.method,
+            filename: filename
+          }
+        else
+          "%s POST 1=1" % url
+        end
+      else
+        raise NotImplementedError, "received a #{request.method.inspect} request, unhandled"
+      end
+    end
+
+    def url_for(uri)
+      path = uri.path
+      #path += ".json" unless path =~ /\./ || path[-1] == '/'
+
+      url = "%{proto}%{host}%{path}" % {
+        proto: protocol_for(uri),
+        host: host_for(uri),
+        path: path
+      }
+
+      url += "?#{uri.query}" if uri.query
+      url
+    end
+  end
+end
+
+Hardy::Siege.start

data/hardy.gemspec

@@ -0,0 +1,24 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'hardy/version'
+
+Gem::Specification.new do |gem|
+  gem.name          = "hardy"
+  gem.version       = Hardy::VERSION
+  gem.authors       = ["Nathaniel Bibler"]
+  gem.email         = ["contact@nathanielbibler.com"]
+  gem.description   = %q{Convert an HTTP Archive (HAR) into a siege URLs file with Content-Type request support.}
+  gem.summary       = %q{Convert HTTP Archive (HAR) files to siege URL files}
+  gem.homepage      = "https://github.com/nbibler/hardy"
+
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+
+  gem.add_dependency 'addressable', '~>2.0'
+  gem.add_dependency 'har'
+  gem.add_dependency 'mime-types', '~>1.0'
+  gem.add_dependency 'thor'
+end

data/lib/hardy/version.rb

@@ -0,0 +1,3 @@
+module Hardy
+  VERSION = "0.0.1"
+end

metadata

@@ -0,0 +1,120 @@
+--- !ruby/object:Gem::Specification
+name: hardy
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Nathaniel Bibler
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-02-15 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: addressable
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '2.0'
+- !ruby/object:Gem::Dependency
+  name: har
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mime-types
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Convert an HTTP Archive (HAR) into a siege URLs file with Content-Type
+  request support.
+email:
+- contact@nathanielbibler.com
+executables:
+- hardy
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- CHANGELOG.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/hardy
+- hardy.gemspec
+- lib/hardy/version.rb
+homepage: https://github.com/nbibler/hardy
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.25
+signing_key: 
+specification_version: 3
+summary: Convert HTTP Archive (HAR) files to siege URL files
+test_files: []