joust 1.0.0

data/LICENSE.txt

@@ -0,0 +1,19 @@
+Copyright (c) 2012 Vertive, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,54 @@
+= Joust - A JSON-RPC Server testing utility
+
+== What is JSON-RPC?
+From the [JSON-RPC Spec](http://www.jsonrpc.org/spec.html):
+
+"JSON-RPC is a stateless, light-weight remote procedure call (RPC) protocol. Primarily this specification defines several data structures and the rules around their processing.
+It is transport agnostic in that the concepts can be used within the same process, over sockets, over http, or in many various message passing environments. It uses JSON (RFC 4627)
+as data format. It is designed to be simple!"
+
+== What is Joust good for?
+There are many JSON-RPC frameworks available for a variety of languages, but it can be difficult to tell which ones truly conform to specifications. Joust is a utility that allows a user or
+developer to determine if a particular framework meets these specifications (and if it does not it enumerates how the framework failed).
+
+== What versions of JSON-RPC specifications can I test for?
+Joust supports testing for JSON-RPC [v1.2](http://jsonrpc.org/historical/jsonrpc12_proposal.html#specification) and [v2.0](http://www.jsonrpc.org/spec.html) (v2.0 is used by default).
+
+== How do I use Joust?
+JSON-RPC tests a server framework by sending predetermined requests and examining the server's response versus the expected response. To use Joust, set up a JSON-RPC Server that supports the
+following methods (and ONLY the following methods):
+
+*   subtract: takes two parameters and returns difference between the first minus the second
+*   subtract_named: takes two named parameters ('minuend' and 'subtrahend') in any order and subtracts minuend minus subtrahend.
+*   update: takes five parameters and does nothing
+*   foobar: takes no parameters and does nothing
+*   sum: takes three parameters and returns their sum
+*   notify_hello: takes one parameter and does nothing
+*   get_data: takes no paramters and returns an array of 'hello' and 5
+*   notify_sum: takes three parameters and does nothing
+
+Examples of a PHP class and a Ruby class with the above methods defined can be found in the example directory of [the Github repo](https://github.com/EvilScott/joust). Note that all of these
+methods are used to test a JSON-RPC v2.0 server, but only a subset of them are used to test v1.2.
+
+After setting this up just point Joust at the server and check out the results:
+
+    joust <JSON-RPC server url>
+
+The results should look something like this:
+
+    v2.0 Positional parameters (1/2) -- PASS
+    v2.0 Positional parameters (2/2) -- PASS
+    v2.0 Named parameters (1/2) -- PASS
+    v2.0 Named parameters (2/2) -- PASS
+    v2.0 Notification (1/2) -- PASS
+    v2.0 Notification (2/2) -- PASS
+    v2.0 Missing method -- PASS
+    v2.0 Invalid JSON -- PASS
+    v2.0 Invalid JSON-RPC -- PASS
+    v2.0 Invalid batch JSON -- PASS
+    v2.0 Empty array -- PASS
+    v2.0 Invalid batch JSON-RPC (1/2) -- PASS
+    v2.0 Invalid batch JSON-RPC (2/2) -- PASS
+    v2.0 Batch call -- PASS
+    v2.0 Batch call (all notifications) -- PASS
+    JSON-RPC v2.0: 15/15 tests passed

data/Rakefile

@@ -0,0 +1,8 @@
+require 'rspec/core/rake_task'
+
+desc "Run all specs"
+RSpec::Core::RakeTask.new(:rspec) do |t|
+  t.pattern = 'spec/**/*_spec.rb'
+end
+
+task :default => :rspec

data/bin/joust

@@ -0,0 +1,42 @@
+#!/usr/bin/env ruby
+# Joust - JSON-RPC Server Specification Testing
+# R. Scott Reis - February 2012
+
+# core joust code
+$:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
+require 'joust'
+
+# check for valid URL
+url = ARGV.first
+begin
+  URI.parse(url)
+rescue
+  puts "URL #{url} does not appear to be valid"
+  exit(1)
+end
+
+# default options
+options = {:version => 'v2.0'}
+
+# command line options
+require 'optparse'
+OptionParser.new do |opts|
+  opts.banner = 'Usage: joust <url> [options]'
+
+  opts.on('-o', '--only-check (v1.2|v2.0)', 'Check only against JSON-RPC Specifications for specified version') do |v|
+    options[:version] = v
+  end
+
+  opts.on_tail("-?", "--help", "Show this message") do
+    puts opts
+    exit
+  end
+
+end.parse!
+
+# specification test cases
+require 'yaml'
+test_cases = YAML.load(File.open(File.join(File.dirname(__FILE__), '..', 'data', 'specifications.yml')))
+
+# run the spec checking
+Joust.run(url, options, test_cases)

data/data/specifications.yml

@@ -0,0 +1,126 @@
+---
+
+v2.0: # http://www.jsonrpc.org/spec.html
+  - "Positional parameters (1/2)":
+    - '{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {result: 19, id: 1}
+  - "Positional parameters (2/2)":
+    - '{"jsonrpc": "2.0", "method": "subtract", "params": [23, 42], "id": 2}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {result: -19, id: 2}
+  - "Named parameters (1/2)":
+    - '{"jsonrpc": "2.0", "method": "subtract_named", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {result: 19, id: 3}
+  - "Named parameters (2/2)":
+    - '{"jsonrpc": "2.0", "method": "subtract_named", "params": {"minuend": 42, "subtrahend": 23}, "id": 4}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {result: 19, id: 4}
+  - "Notification (1/2)":
+    - '{"jsonrpc": "2.0", "method": "update", "params": [1,2,3,4,5]}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {}
+  - "Notification (2/2)":
+    - '{"jsonrpc": "2.0", "method": "foobar"}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {}
+  - "Missing method":
+    - '{"jsonrpc": "2.0", "method": "notthere", "id": 5}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {error: {code: -32601, message: "Method not found."}, id: 5}
+  - "Invalid JSON":
+    - '{"jsonrpc": "2.0", "method": "foobar, "params": "bar", "baz]'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {error: {code: -32700, message: "Parse error."}, id: ~}
+  - "Invalid JSON-RPC":
+    - '{"jsonrpc": "2.0", "method": 1, "params": "bar"}'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+  - "Invalid batch JSON":
+    - '[ {"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": "1"},{"jsonrpc": "2.0", "method" ]'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {error: {code: -32700, message: "Parse error."}, id: ~}
+  - "Empty array":
+    - '[]'
+    - !ruby/object:YAML::Joust::Version20
+      expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+  - "Invalid batch JSON-RPC (1/2)":
+    - '[1]'
+    -
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+  - "Invalid batch JSON-RPC (2/2)":
+    - '[1,2,3]'
+    -
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+  - "Batch call":
+    - >- [{"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": 1},
+      {"jsonrpc": "2.0", "method": "notify_hello", "params": [7]},
+      {"jsonrpc": "2.0", "method": "subtract", "params": [42,23], "id": 2},
+      {"foo": "boo"},
+      {"jsonrpc": "2.0", "method": "notthere", "params": {"name": "myself"}, "id": 5},
+      {"jsonrpc": "2.0", "method": "get_data", "id": 9}]
+    -
+      - !ruby/object:YAML::Joust::Version20
+        expected: {result: 7, id: 1}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {result: 19, id: 2}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {error: {code: -32601, message: "Method not found."}, id: 5}
+      - !ruby/object:YAML::Joust::Version20
+        expected: {result: ["hello", 5], id: 9}
+  - "Batch call (all notifications)":
+    - >- [{"jsonrpc": "2.0", "method": "notify_sum", "params": [1,2,4]},
+      {"jsonrpc": "2.0", "method": "notify_hello", "params": [7]}]
+    - !ruby/object:YAML::Joust::Version20
+      expected: {}
+
+v1.2: # http://jsonrpc.org/historical/jsonrpc12_proposal.html#specification
+  - "Positional parameters (1/2)":
+    - '{"method": "subtract", "params": [42, 23], "id": 1}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result: 19, error:~, id: 1}
+  - "Positional parameters (2/2)":
+    - '{"method": "subtract", "params": [23, 42], "id": 2}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result: -19, error:~, id: 2}
+  - "Named parameters (1/2)":
+    - '{"method": "subtract_named", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result: 19, error:~, id: 3}
+  - "Named parameters (2/2)":
+    - '{"method": "subtract_named", "params": {"minuend": 42, "subtrahend": 23}, "id": 4}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result: 19, error:~, id: 4}
+  - "Notification (1/2)":
+    - '{"method": "update", "params": [1,2,3,4,5], "id": null}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {}
+  - "Notification (2/2)":
+    - '{"method": "foobar", error:~, "id": null}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {}
+  - "Missing method":
+    - '{ "method": "notthere", "id": 5}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result:~, error: {code: -32601, message: "Procedure not found."}, id: 5}
+  - "Invalid JSON":
+    - '{"method": "foobar, "params": "bar", "baz]'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result:~, error: {code: -32700, message: "Parse error."}, id: ~}
+  - "Invalid JSON-RPC (1/2)":
+    - '[1,2,3]'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result:~, error: {code: -32600, message: "Invalid JSON-RPC."}, id: ~}
+  - "Invalid JSON-RPC (2/2)":
+    - '{"method": 1, "params": "bar"}'
+    - !ruby/object:YAML::Joust::Version12
+      expected: {result:~, error: {code: -32600, message: "Invalid JSON-RPC."}, id: ~}

data/lib/joust.rb

@@ -0,0 +1,45 @@
+require 'json'
+require 'rest-client'
+require File.join('joust','expected_response.rb')
+require File.join('joust','expected_response','version_12.rb')
+require File.join('joust','expected_response','version_20.rb')
+
+class Joust
+
+  # run each version spec and return results
+  def self.run(url, options, test_cases)
+    specs = self.new(url, test_cases[options[:version]], options[:version])
+    puts specs.check
+  end
+
+  # initialize an instance to test a specific version
+  def initialize(url, test_cases, version)
+    @url, @test_cases, @version = url, test_cases, version
+  end
+
+  # run specification checks and return results
+  def check
+    num_tests = passed_tests = 0
+    results = @test_cases.map(&:flatten).inject([]) do |results, test|
+      test_name, sent, expected = test.first, test.last.first, test.last.last
+      resp = RestClient.post(@url, sent, :content_type => 'application/json')
+      num_tests += 1
+      json_hash = JSON.parse(resp.body) rescue {}
+      if expected.is_a?(Array)
+        pass = expected.inject(true) { |pass, exp| pass = false unless exp.match?(json_hash.shift); pass }
+      else
+        pass = expected.match?(json_hash)
+      end
+      if !!pass
+        passed_tests += 1
+        result_output = 'PASS'
+      else
+        result_output = 'FAIL: ' + resp.body
+      end
+      results << "#{@version} #{test_name} -- #{result_output}"
+      results
+    end
+    results << "JSON-RPC #{@version}: #{passed_tests}/#{num_tests} tests passed"
+  end
+
+end

data/lib/joust/expected_response.rb

@@ -0,0 +1,20 @@
+class Joust
+  class ExpectedResponse
+
+    def match?(json_hash)
+      ['result', 'error', 'id'].each { |key| @expected[key] = :missing if !@expected.has_key?(key) }
+      if json_hash.empty?
+        return false unless @expected.values == [:missing, :missing, :missing]
+        return true
+      end
+      @expected.each do |key, val|
+        next if val == :missing && !json_hash.has_key?(key)
+        return false unless json_hash[key] == val
+      end
+      true
+      rescue Exception
+        false
+    end
+
+  end
+end

data/lib/joust/expected_response/version_12.rb

@@ -0,0 +1,10 @@
+class Joust
+  class Version12 < ExpectedResponse
+
+    def match?(json_hash)
+      return false unless super
+      json_hash.has_key?('result') && json_hash.has_key?('error') && json_hash.has_key?('id')
+    end
+
+  end
+end

data/lib/joust/expected_response/version_20.rb

@@ -0,0 +1,10 @@
+class Joust
+  class Version20 < ExpectedResponse
+
+    def match?(json_hash)
+      return false unless super
+      (json_hash['jsonrpc'] == '2.0' && (!!json_hash['result'] ^ !!json_hash['error'])) || json_hash.empty?
+    end
+
+  end
+end

data/spec/joust_spec.rb

@@ -0,0 +1,41 @@
+require 'spec_helper'
+
+describe Joust do
+
+  describe '.run' do
+    context 'given a target url, options, and a test case file' do
+      it 'creates a new instance if joust and runs the tests' do
+        fake_joust = double('joust')
+        fake_joust.should_receive(:check)
+        Joust.should_receive(:new).and_return(fake_joust)
+        Joust.run('http://url', {:foo => :bar}, {:version => :foo})
+      end
+    end
+  end
+
+  describe '#initialize' do
+    context 'given a target url, options, and a test case file' do
+      it 'creates a new instance of Joust' do
+        joust = Joust.new('http://url', {:foo => :bar}, {:version => :foo})
+        joust.should be_instance_of Joust
+      end
+    end
+  end
+
+  describe '#check' do
+    it 'runs the test cases' do
+      fake_resp = double('resp')
+      fake_resp.stub!(:body)
+      content_type = {:content_type => 'application/json'}
+      RestClient.should_receive(:post).with('url', 'sent', content_type).and_return(fake_resp)
+      fake_expected = double('expected')
+      fake_expected.should_receive(:is_a?).with(Array).and_return(false)
+      fake_expected.should_receive(:match?).with({}).and_return(true)
+      fake_test_cases = [{'test' => ['sent', fake_expected]}]
+      joust = Joust.new('url', fake_test_cases, 'v3.14')
+      results = joust.check
+      results.last.should == "JSON-RPC v3.14: 1/1 tests passed"
+    end
+  end
+
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+require 'rspec'
+require File.join(File.dirname(__FILE__),'..','lib','joust.rb')

metadata

@@ -0,0 +1,63 @@
+--- !ruby/object:Gem::Specification
+name: joust
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+  prerelease: 
+platform: ruby
+authors:
+- R. Scott Reis
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2012-03-14 00:00:00.000000000Z
+dependencies: []
+description: 
+email: 
+executables:
+- joust
+extensions: []
+extra_rdoc_files:
+- README.rdoc
+files:
+- LICENSE.txt
+- README.rdoc
+- Rakefile
+- bin/joust
+- data/specifications.yml
+- lib/joust/expected_response.rb
+- lib/joust/expected_response/version_12.rb
+- lib/joust/expected_response/version_20.rb
+- lib/joust.rb
+- spec/spec_helper.rb
+- spec/joust_spec.rb
+homepage: https://github.com/EvilScott/joust
+licenses: []
+post_install_message: 
+rdoc_options:
+- -m
+- README.rdoc
+- -t
+- Joust
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.18
+signing_key: 
+specification_version: 3
+summary: JSON-RPC Server Specification Compatibility Testing
+test_files:
+- spec/spec_helper.rb
+- spec/joust_spec.rb