joust 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,19 @@
1
+ Copyright (c) 2012 Vertive, Inc.
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy
4
+ of this software and associated documentation files (the "Software"), to deal
5
+ in the Software without restriction, including without limitation the rights
6
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7
+ copies of the Software, and to permit persons to whom the Software is
8
+ furnished to do so, subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in
11
+ all copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19
+ THE SOFTWARE.
@@ -0,0 +1,54 @@
1
+ = Joust - A JSON-RPC Server testing utility
2
+
3
+ == What is JSON-RPC?
4
+ From the [JSON-RPC Spec](http://www.jsonrpc.org/spec.html):
5
+
6
+ "JSON-RPC is a stateless, light-weight remote procedure call (RPC) protocol. Primarily this specification defines several data structures and the rules around their processing.
7
+ It is transport agnostic in that the concepts can be used within the same process, over sockets, over http, or in many various message passing environments. It uses JSON (RFC 4627)
8
+ as data format. It is designed to be simple!"
9
+
10
+ == What is Joust good for?
11
+ There are many JSON-RPC frameworks available for a variety of languages, but it can be difficult to tell which ones truly conform to specifications. Joust is a utility that allows a user or
12
+ developer to determine if a particular framework meets these specifications (and if it does not it enumerates how the framework failed).
13
+
14
+ == What versions of JSON-RPC specifications can I test for?
15
+ Joust supports testing for JSON-RPC [v1.2](http://jsonrpc.org/historical/jsonrpc12_proposal.html#specification) and [v2.0](http://www.jsonrpc.org/spec.html) (v2.0 is used by default).
16
+
17
+ == How do I use Joust?
18
+ JSON-RPC tests a server framework by sending predetermined requests and examining the server's response versus the expected response. To use Joust, set up a JSON-RPC Server that supports the
19
+ following methods (and ONLY the following methods):
20
+
21
+ * subtract: takes two parameters and returns difference between the first minus the second
22
+ * subtract_named: takes two named parameters ('minuend' and 'subtrahend') in any order and subtracts minuend minus subtrahend.
23
+ * update: takes five parameters and does nothing
24
+ * foobar: takes no parameters and does nothing
25
+ * sum: takes three parameters and returns their sum
26
+ * notify_hello: takes one parameter and does nothing
27
+ * get_data: takes no paramters and returns an array of 'hello' and 5
28
+ * notify_sum: takes three parameters and does nothing
29
+
30
+ Examples of a PHP class and a Ruby class with the above methods defined can be found in the example directory of [the Github repo](https://github.com/EvilScott/joust). Note that all of these
31
+ methods are used to test a JSON-RPC v2.0 server, but only a subset of them are used to test v1.2.
32
+
33
+ After setting this up just point Joust at the server and check out the results:
34
+
35
+ joust <JSON-RPC server url>
36
+
37
+ The results should look something like this:
38
+
39
+ v2.0 Positional parameters (1/2) -- PASS
40
+ v2.0 Positional parameters (2/2) -- PASS
41
+ v2.0 Named parameters (1/2) -- PASS
42
+ v2.0 Named parameters (2/2) -- PASS
43
+ v2.0 Notification (1/2) -- PASS
44
+ v2.0 Notification (2/2) -- PASS
45
+ v2.0 Missing method -- PASS
46
+ v2.0 Invalid JSON -- PASS
47
+ v2.0 Invalid JSON-RPC -- PASS
48
+ v2.0 Invalid batch JSON -- PASS
49
+ v2.0 Empty array -- PASS
50
+ v2.0 Invalid batch JSON-RPC (1/2) -- PASS
51
+ v2.0 Invalid batch JSON-RPC (2/2) -- PASS
52
+ v2.0 Batch call -- PASS
53
+ v2.0 Batch call (all notifications) -- PASS
54
+ JSON-RPC v2.0: 15/15 tests passed
@@ -0,0 +1,8 @@
1
+ require 'rspec/core/rake_task'
2
+
3
+ desc "Run all specs"
4
+ RSpec::Core::RakeTask.new(:rspec) do |t|
5
+ t.pattern = 'spec/**/*_spec.rb'
6
+ end
7
+
8
+ task :default => :rspec
@@ -0,0 +1,42 @@
1
+ #!/usr/bin/env ruby
2
+ # Joust - JSON-RPC Server Specification Testing
3
+ # R. Scott Reis - February 2012
4
+
5
+ # core joust code
6
+ $:.unshift File.join(File.dirname(__FILE__), '..', 'lib')
7
+ require 'joust'
8
+
9
+ # check for valid URL
10
+ url = ARGV.first
11
+ begin
12
+ URI.parse(url)
13
+ rescue
14
+ puts "URL #{url} does not appear to be valid"
15
+ exit(1)
16
+ end
17
+
18
+ # default options
19
+ options = {:version => 'v2.0'}
20
+
21
+ # command line options
22
+ require 'optparse'
23
+ OptionParser.new do |opts|
24
+ opts.banner = 'Usage: joust <url> [options]'
25
+
26
+ opts.on('-o', '--only-check (v1.2|v2.0)', 'Check only against JSON-RPC Specifications for specified version') do |v|
27
+ options[:version] = v
28
+ end
29
+
30
+ opts.on_tail("-?", "--help", "Show this message") do
31
+ puts opts
32
+ exit
33
+ end
34
+
35
+ end.parse!
36
+
37
+ # specification test cases
38
+ require 'yaml'
39
+ test_cases = YAML.load(File.open(File.join(File.dirname(__FILE__), '..', 'data', 'specifications.yml')))
40
+
41
+ # run the spec checking
42
+ Joust.run(url, options, test_cases)
@@ -0,0 +1,126 @@
1
+ ---
2
+
3
+ v2.0: # http://www.jsonrpc.org/spec.html
4
+ - "Positional parameters (1/2)":
5
+ - '{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}'
6
+ - !ruby/object:YAML::Joust::Version20
7
+ expected: {result: 19, id: 1}
8
+ - "Positional parameters (2/2)":
9
+ - '{"jsonrpc": "2.0", "method": "subtract", "params": [23, 42], "id": 2}'
10
+ - !ruby/object:YAML::Joust::Version20
11
+ expected: {result: -19, id: 2}
12
+ - "Named parameters (1/2)":
13
+ - '{"jsonrpc": "2.0", "method": "subtract_named", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}'
14
+ - !ruby/object:YAML::Joust::Version20
15
+ expected: {result: 19, id: 3}
16
+ - "Named parameters (2/2)":
17
+ - '{"jsonrpc": "2.0", "method": "subtract_named", "params": {"minuend": 42, "subtrahend": 23}, "id": 4}'
18
+ - !ruby/object:YAML::Joust::Version20
19
+ expected: {result: 19, id: 4}
20
+ - "Notification (1/2)":
21
+ - '{"jsonrpc": "2.0", "method": "update", "params": [1,2,3,4,5]}'
22
+ - !ruby/object:YAML::Joust::Version20
23
+ expected: {}
24
+ - "Notification (2/2)":
25
+ - '{"jsonrpc": "2.0", "method": "foobar"}'
26
+ - !ruby/object:YAML::Joust::Version20
27
+ expected: {}
28
+ - "Missing method":
29
+ - '{"jsonrpc": "2.0", "method": "notthere", "id": 5}'
30
+ - !ruby/object:YAML::Joust::Version20
31
+ expected: {error: {code: -32601, message: "Method not found."}, id: 5}
32
+ - "Invalid JSON":
33
+ - '{"jsonrpc": "2.0", "method": "foobar, "params": "bar", "baz]'
34
+ - !ruby/object:YAML::Joust::Version20
35
+ expected: {error: {code: -32700, message: "Parse error."}, id: ~}
36
+ - "Invalid JSON-RPC":
37
+ - '{"jsonrpc": "2.0", "method": 1, "params": "bar"}'
38
+ - !ruby/object:YAML::Joust::Version20
39
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
40
+ - "Invalid batch JSON":
41
+ - '[ {"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": "1"},{"jsonrpc": "2.0", "method" ]'
42
+ - !ruby/object:YAML::Joust::Version20
43
+ expected: {error: {code: -32700, message: "Parse error."}, id: ~}
44
+ - "Empty array":
45
+ - '[]'
46
+ - !ruby/object:YAML::Joust::Version20
47
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
48
+ - "Invalid batch JSON-RPC (1/2)":
49
+ - '[1]'
50
+ -
51
+ - !ruby/object:YAML::Joust::Version20
52
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
53
+ - "Invalid batch JSON-RPC (2/2)":
54
+ - '[1,2,3]'
55
+ -
56
+ - !ruby/object:YAML::Joust::Version20
57
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
58
+ - !ruby/object:YAML::Joust::Version20
59
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
60
+ - !ruby/object:YAML::Joust::Version20
61
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
62
+ - "Batch call":
63
+ - >- [{"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": 1},
64
+ {"jsonrpc": "2.0", "method": "notify_hello", "params": [7]},
65
+ {"jsonrpc": "2.0", "method": "subtract", "params": [42,23], "id": 2},
66
+ {"foo": "boo"},
67
+ {"jsonrpc": "2.0", "method": "notthere", "params": {"name": "myself"}, "id": 5},
68
+ {"jsonrpc": "2.0", "method": "get_data", "id": 9}]
69
+ -
70
+ - !ruby/object:YAML::Joust::Version20
71
+ expected: {result: 7, id: 1}
72
+ - !ruby/object:YAML::Joust::Version20
73
+ expected: {result: 19, id: 2}
74
+ - !ruby/object:YAML::Joust::Version20
75
+ expected: {error: {code: -32600, message: "Invalid Request."}, id: ~}
76
+ - !ruby/object:YAML::Joust::Version20
77
+ expected: {error: {code: -32601, message: "Method not found."}, id: 5}
78
+ - !ruby/object:YAML::Joust::Version20
79
+ expected: {result: ["hello", 5], id: 9}
80
+ - "Batch call (all notifications)":
81
+ - >- [{"jsonrpc": "2.0", "method": "notify_sum", "params": [1,2,4]},
82
+ {"jsonrpc": "2.0", "method": "notify_hello", "params": [7]}]
83
+ - !ruby/object:YAML::Joust::Version20
84
+ expected: {}
85
+
86
+ v1.2: # http://jsonrpc.org/historical/jsonrpc12_proposal.html#specification
87
+ - "Positional parameters (1/2)":
88
+ - '{"method": "subtract", "params": [42, 23], "id": 1}'
89
+ - !ruby/object:YAML::Joust::Version12
90
+ expected: {result: 19, error:~, id: 1}
91
+ - "Positional parameters (2/2)":
92
+ - '{"method": "subtract", "params": [23, 42], "id": 2}'
93
+ - !ruby/object:YAML::Joust::Version12
94
+ expected: {result: -19, error:~, id: 2}
95
+ - "Named parameters (1/2)":
96
+ - '{"method": "subtract_named", "params": {"subtrahend": 23, "minuend": 42}, "id": 3}'
97
+ - !ruby/object:YAML::Joust::Version12
98
+ expected: {result: 19, error:~, id: 3}
99
+ - "Named parameters (2/2)":
100
+ - '{"method": "subtract_named", "params": {"minuend": 42, "subtrahend": 23}, "id": 4}'
101
+ - !ruby/object:YAML::Joust::Version12
102
+ expected: {result: 19, error:~, id: 4}
103
+ - "Notification (1/2)":
104
+ - '{"method": "update", "params": [1,2,3,4,5], "id": null}'
105
+ - !ruby/object:YAML::Joust::Version12
106
+ expected: {}
107
+ - "Notification (2/2)":
108
+ - '{"method": "foobar", error:~, "id": null}'
109
+ - !ruby/object:YAML::Joust::Version12
110
+ expected: {}
111
+ - "Missing method":
112
+ - '{ "method": "notthere", "id": 5}'
113
+ - !ruby/object:YAML::Joust::Version12
114
+ expected: {result:~, error: {code: -32601, message: "Procedure not found."}, id: 5}
115
+ - "Invalid JSON":
116
+ - '{"method": "foobar, "params": "bar", "baz]'
117
+ - !ruby/object:YAML::Joust::Version12
118
+ expected: {result:~, error: {code: -32700, message: "Parse error."}, id: ~}
119
+ - "Invalid JSON-RPC (1/2)":
120
+ - '[1,2,3]'
121
+ - !ruby/object:YAML::Joust::Version12
122
+ expected: {result:~, error: {code: -32600, message: "Invalid JSON-RPC."}, id: ~}
123
+ - "Invalid JSON-RPC (2/2)":
124
+ - '{"method": 1, "params": "bar"}'
125
+ - !ruby/object:YAML::Joust::Version12
126
+ expected: {result:~, error: {code: -32600, message: "Invalid JSON-RPC."}, id: ~}
@@ -0,0 +1,45 @@
1
+ require 'json'
2
+ require 'rest-client'
3
+ require File.join('joust','expected_response.rb')
4
+ require File.join('joust','expected_response','version_12.rb')
5
+ require File.join('joust','expected_response','version_20.rb')
6
+
7
+ class Joust
8
+
9
+ # run each version spec and return results
10
+ def self.run(url, options, test_cases)
11
+ specs = self.new(url, test_cases[options[:version]], options[:version])
12
+ puts specs.check
13
+ end
14
+
15
+ # initialize an instance to test a specific version
16
+ def initialize(url, test_cases, version)
17
+ @url, @test_cases, @version = url, test_cases, version
18
+ end
19
+
20
+ # run specification checks and return results
21
+ def check
22
+ num_tests = passed_tests = 0
23
+ results = @test_cases.map(&:flatten).inject([]) do |results, test|
24
+ test_name, sent, expected = test.first, test.last.first, test.last.last
25
+ resp = RestClient.post(@url, sent, :content_type => 'application/json')
26
+ num_tests += 1
27
+ json_hash = JSON.parse(resp.body) rescue {}
28
+ if expected.is_a?(Array)
29
+ pass = expected.inject(true) { |pass, exp| pass = false unless exp.match?(json_hash.shift); pass }
30
+ else
31
+ pass = expected.match?(json_hash)
32
+ end
33
+ if !!pass
34
+ passed_tests += 1
35
+ result_output = 'PASS'
36
+ else
37
+ result_output = 'FAIL: ' + resp.body
38
+ end
39
+ results << "#{@version} #{test_name} -- #{result_output}"
40
+ results
41
+ end
42
+ results << "JSON-RPC #{@version}: #{passed_tests}/#{num_tests} tests passed"
43
+ end
44
+
45
+ end
@@ -0,0 +1,20 @@
1
+ class Joust
2
+ class ExpectedResponse
3
+
4
+ def match?(json_hash)
5
+ ['result', 'error', 'id'].each { |key| @expected[key] = :missing if !@expected.has_key?(key) }
6
+ if json_hash.empty?
7
+ return false unless @expected.values == [:missing, :missing, :missing]
8
+ return true
9
+ end
10
+ @expected.each do |key, val|
11
+ next if val == :missing && !json_hash.has_key?(key)
12
+ return false unless json_hash[key] == val
13
+ end
14
+ true
15
+ rescue Exception
16
+ false
17
+ end
18
+
19
+ end
20
+ end
@@ -0,0 +1,10 @@
1
+ class Joust
2
+ class Version12 < ExpectedResponse
3
+
4
+ def match?(json_hash)
5
+ return false unless super
6
+ json_hash.has_key?('result') && json_hash.has_key?('error') && json_hash.has_key?('id')
7
+ end
8
+
9
+ end
10
+ end
@@ -0,0 +1,10 @@
1
+ class Joust
2
+ class Version20 < ExpectedResponse
3
+
4
+ def match?(json_hash)
5
+ return false unless super
6
+ (json_hash['jsonrpc'] == '2.0' && (!!json_hash['result'] ^ !!json_hash['error'])) || json_hash.empty?
7
+ end
8
+
9
+ end
10
+ end
@@ -0,0 +1,41 @@
1
+ require 'spec_helper'
2
+
3
+ describe Joust do
4
+
5
+ describe '.run' do
6
+ context 'given a target url, options, and a test case file' do
7
+ it 'creates a new instance if joust and runs the tests' do
8
+ fake_joust = double('joust')
9
+ fake_joust.should_receive(:check)
10
+ Joust.should_receive(:new).and_return(fake_joust)
11
+ Joust.run('http://url', {:foo => :bar}, {:version => :foo})
12
+ end
13
+ end
14
+ end
15
+
16
+ describe '#initialize' do
17
+ context 'given a target url, options, and a test case file' do
18
+ it 'creates a new instance of Joust' do
19
+ joust = Joust.new('http://url', {:foo => :bar}, {:version => :foo})
20
+ joust.should be_instance_of Joust
21
+ end
22
+ end
23
+ end
24
+
25
+ describe '#check' do
26
+ it 'runs the test cases' do
27
+ fake_resp = double('resp')
28
+ fake_resp.stub!(:body)
29
+ content_type = {:content_type => 'application/json'}
30
+ RestClient.should_receive(:post).with('url', 'sent', content_type).and_return(fake_resp)
31
+ fake_expected = double('expected')
32
+ fake_expected.should_receive(:is_a?).with(Array).and_return(false)
33
+ fake_expected.should_receive(:match?).with({}).and_return(true)
34
+ fake_test_cases = [{'test' => ['sent', fake_expected]}]
35
+ joust = Joust.new('url', fake_test_cases, 'v3.14')
36
+ results = joust.check
37
+ results.last.should == "JSON-RPC v3.14: 1/1 tests passed"
38
+ end
39
+ end
40
+
41
+ end
@@ -0,0 +1,2 @@
1
+ require 'rspec'
2
+ require File.join(File.dirname(__FILE__),'..','lib','joust.rb')
metadata ADDED
@@ -0,0 +1,63 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: joust
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - R. Scott Reis
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2012-03-14 00:00:00.000000000Z
13
+ dependencies: []
14
+ description:
15
+ email:
16
+ executables:
17
+ - joust
18
+ extensions: []
19
+ extra_rdoc_files:
20
+ - README.rdoc
21
+ files:
22
+ - LICENSE.txt
23
+ - README.rdoc
24
+ - Rakefile
25
+ - bin/joust
26
+ - data/specifications.yml
27
+ - lib/joust/expected_response.rb
28
+ - lib/joust/expected_response/version_12.rb
29
+ - lib/joust/expected_response/version_20.rb
30
+ - lib/joust.rb
31
+ - spec/spec_helper.rb
32
+ - spec/joust_spec.rb
33
+ homepage: https://github.com/EvilScott/joust
34
+ licenses: []
35
+ post_install_message:
36
+ rdoc_options:
37
+ - -m
38
+ - README.rdoc
39
+ - -t
40
+ - Joust
41
+ require_paths:
42
+ - lib
43
+ required_ruby_version: !ruby/object:Gem::Requirement
44
+ none: false
45
+ requirements:
46
+ - - ! '>='
47
+ - !ruby/object:Gem::Version
48
+ version: '0'
49
+ required_rubygems_version: !ruby/object:Gem::Requirement
50
+ none: false
51
+ requirements:
52
+ - - ! '>='
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ requirements: []
56
+ rubyforge_project:
57
+ rubygems_version: 1.8.18
58
+ signing_key:
59
+ specification_version: 3
60
+ summary: JSON-RPC Server Specification Compatibility Testing
61
+ test_files:
62
+ - spec/spec_helper.rb
63
+ - spec/joust_spec.rb