webspicy 0.15.8 → 0.16.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e1d234226f15d51777e61c010557ed1691660613cd87ba11ce19844b2f7cc812
-  data.tar.gz: 40fcd4639ce847f7f69f3471ef9676c64e8e6be489388563d5abd51ca1bf39d7
+  metadata.gz: 70bfa6517173d1f121b9ebe968ac5970e41b2f6846384b152d288eb2f4913cb7
+  data.tar.gz: 0e5b728db6ecd9eb3ccb42d166317911a03092d41cf4778f1058960aa0e92bab
 SHA512:
-  metadata.gz: c4d0ec9a0567955bac41c5ee887164e783799e2baf66966fabff404147376c29192a4b28090c7d816a20fd15bc7e470b20136e253b9b1a7d663ac57de4958d99
-  data.tar.gz: 4ddccde82357c81066108133b0e5b63ea8b5a0b714556b56b4d4743ba26aaaf17a9187633061dc2f721b88833cdfb536f795d8256aa158723c4fa5a255957144
+  metadata.gz: 51e6fbe8ad57cb2795435a3c194e142d1f5e822289be923aeebfd9c89ef9440a6fd7dea2fb58bb9e8d8bdd490b1025201dfb12b4d000d7d6604e422de1007dcf
+  data.tar.gz: 3fe5b84bd684564321712412c2bc0f918944d22dbe665ea119d0a478d181a6ac9483de2c1352e8b0f98b484e1cd361f0193afcf42c6ee6c0095811feec86c555

data/README.md

@@ -1,46 +1,93 @@
 # Webspicy
 
-A description, specification and test framework for web services seen as black
-box software operations.
+A specification and test framework for web services seen as black-box software
+operations. Webspicy yields a better test coverage for a smaller testing effort,
+because software quality matters.
+
+See webspicy in action and make the tutorial on https://yourbackendisbroken.dev
 
 ## Features
 
-* Declarative description of RESTful web-services + their tests
+* Declarative specification of HTTP web services + their tests
+
+* Framework/language agnostic: `webspicy` is written in Ruby, but can be used
+  to test web services for backends written in any language / framework.
+
+* Black box testing: `webspicy` focuses on web services seen as blackboxes. It
+  has no knowledge of the implementation, and focuses on HTTP and input/output
+  data instead. Investing in such testing makes those tests more stable and
+  your API design better.
 
-* Framework/language agnostic: Webspicy is written in Ruby, but can be used to
-  test web services in any language / framework.
+* Formal and human-friendly data schema with strong data matching semantics,
+  thanks to finitio.io
 
-* Black box testing: Webspicy focuses on web services seen as blackboxes. It has
-  no knowledge of the implementation, and focuses on HTTP and input/output data
-  instead. Investing in such testing makes those tests more stable and your API
-  better.
+* Test instrumentation and generation, based on PRE & POST contracts.
 
-* Test instrumentation and generation, based on declarative PRE & POST
-  conditions.
+* Extra goodness for Rubyists: being written in ruby, `webspicy` also supports
+  testing Rack applications directly (through rack/test), which boosts the
+  test suite.
 
-* Extra goodness for Rubyists: being written in ruby, Webspicy also supports
-  testing Rack applications directly (through rack/test)
+* Extra goodies: when a specification is written, it can also be used for
+  mocking the API, generating an openapi file, etc.
 
-## Getting started
+## Getting started with the commandline
 
-Please have a look at the example first. It contains a simple Sinatra application
-with GET and POST restful services tested with the framework. The Rakefile contains
-the necessary tasks to run those tests.
+To install webspicy on your developer computer, install ruby then:
 
-## Executing the tests using the command line
+```
+gem install webspicy
+```
 
-Please have a look at the examples first. Then:
+Then execute webspicy help to see the options.
 
 ```
 webspicy --help
-webspicy path/to/your/config.rb
 ```
 
-## Using the docker image
+## Using the docker image(s)
+
+If you just want to play with the commandline tool without having to
+install ruby & webspicy, you can use the docker image we provide for
+the commandline:
+
+```
+docker run enspirit/webspicy --help
+```
+
+If you have a specification & test suite somewhere, an easy way to run the
+whole suite (or integrate it in your continuous integration pipeline) is
+to use our `:tester` docker image. Just mount your test suite as a volume
+in `/home/app` and you are good to go:
 
-A docker image running your tests is provided. It expects a volume with the
-tests to run put in `/home/app`.
+```
+docker run -v path/to/tests:/formalspec enspirit/webspicy-tester
+```
+
+If your plan is to test a backend that runs on your own machine (vs.
+publicly on the Internet or inside the default docker network),
+you will need to add some networking option, as shown below. Please
+refer to Docker documentation.
 
 ```
-docker run enspirit/webspicy -v path/to/tests:/home/app
+docker run -v path/to/tests:/formalspec --network=host enspirit/webspicy-tester
 ```
+
+## Contributing
+
+Please use github issues for questions and bugs, and pull requests for
+submitting improvement proposals and new features.
+
+## Contributors
+
+* Bernard Lambeau (blambeau)
+* Louis Lambeau (llambeau)
+* Yoann Guyot (ygu)
+* Felix Holmgren (@felixyz)
+
+Enspirit (https://enspirit.be) and Klaro App (https://klaro.cards) are
+both actively using, contributing and funding work on this library.
+Please contact Bernard Lambeau for any question.
+
+## Licence
+
+Webspicy is distributed under a MIT Licence, by Enspirit SRL.

data/bin/webspicy

@@ -1,15 +1,25 @@
 #!/usr/bin/env ruby
-#/ Document and test RESTful webservices
-#/ (c) Enspirit SPRL. Distributed under MIT licence.
-#/ Usage: webspicy [options] path/to/config.rb
+#/
+#/ Document & test web services as black-box operations.
+#/ (c) Enspirit SRL. Distributed under MIT licence.
+#/
+#/ Usage: webspicy [options] URL                 # run default test againt url
+#/ Usage: webspicy [options] path/to/config.rb   # run whole test suite
+#/ Usage: webspicy [options] path/to/spec.yml    # run a single test
+#/
+#/ Options:
+#/ -h, --help       Show this help message
+#/ -v, --version    Show webspicy version
+#/     --debug      Same as LOG_LEVEL=DEBUG, takes precedence
 #/
 #/ Recognized env vars:
-#/ - ROBUST=no                    do not execute counterexamples
+#/ - ROBUST=no|only|generated     execute no counterexamples (only them, only generated ones)
 #/ - LOG_LEVEL=INFO|DEBUG|...     set log level to the appropriate level
 #/ - RESOURCE=...                 restrict to .yml test files matching the pattern
-#/ - METHOD=GET|POST|DELETE...    execute only tests matching the verb
+#/ - METHOD=GET|POST|DELETE...    execute only tests matching the HTTP verb
 #/ - TAG=...                      execute only tests matching the given tag
 #/ - FAILFAST=yes|no              stop executing tests on first failure
+#/
 require 'webspicy'
 require 'webspicy/tester'
 require 'optparse'
@@ -20,19 +30,25 @@ def showhelp
 end
 
 ARGV.options do |opts|
-  opts.on_tail("-h", "--help"){ showhelp }
+  opts.on_tail("-h", "--help"){
+    showhelp
+  }
+  opts.on("-v", "--version"){
+    puts "webspicy v#{Webspicy::VERSION}, (c) Enspirit SRL"
+    exit(0)
+  }
+  opts.on("--debug") {
+    ENV['LOG_LEVEL'] = 'DEBUG'
+    Webspicy::LOGGER.level = Logger.const_get(ENV['LOG_LEVEL'])
+  }
   opts.parse!
 end
 
 if ARGV.size != 1
   showhelp
   exit(-1)
-elsif (p = Path(ARGV.first)).exists?
-  config = Webspicy::Configuration.dress(p)
-  res = Webspicy::Tester.new(config).call
-  abort("Some tests failed") unless res == 0
-else
-  puts "No such file or directory `#{ARGV.first}`"
-  showhelp
-  exit(-1)
 end
+
+config = Webspicy::Configuration.dress(ARGV[0])
+res = Webspicy::Tester.new(config).call
+abort("Some tests failed") unless res == 0

data/examples/restful/Gemfile.lock

@@ -1,12 +1,14 @@
 PATH
   remote: ../..
   specs:
-    webspicy (0.15.8)
-      finitio (>= 0.8.0)
+    webspicy (0.16.0)
+      colorize (~> 0.8.1)
+      finitio (~> 0.9.0)
       http (>= 2)
       mustermann (~> 1.0)
       mustermann-contrib
-      path (>= 1.3)
+      openapi3_parser (~> 0.8.2)
+      path (~> 2.0)
       rack-robustness (~> 1.1, >= 1.1.0)
       rack-test (~> 0.6.3)
       rspec (~> 3.7)
@@ -18,6 +20,10 @@ GEM
     addressable (2.7.0)
       public_suffix (>= 2.0.2, < 5.0)
     citrus (3.0.2)
+    colorize (0.8.1)
+    commonmarker (0.21.0)
+      ruby-enum (~> 0.5)
+    concurrent-ruby (1.1.7)
     diff-lcs (1.4.4)
     domain_name (0.5.20190701)
       unf (>= 0.0.5, < 1.0.0)
@@ -25,7 +31,7 @@ GEM
     ffi-compiler (1.0.1)
       ffi (>= 1.0.0)
       rake
-    finitio (0.8.0)
+    finitio (0.9.0)
       citrus (>= 2.4, < 4.0)
     hansi (0.2.0)
     http (4.4.1)
@@ -36,42 +42,50 @@ GEM
     http-cookie (1.0.3)
       domain_name (~> 0.5)
     http-form_data (2.3.0)
-    http-parser (1.2.1)
-      ffi-compiler (>= 1.0, < 2.0)
+    http-parser (1.2.2)
+      ffi-compiler
+    i18n (1.8.5)
+      concurrent-ruby (~> 1.0)
     mustermann (1.1.1)
       ruby2_keywords (~> 0.0.1)
     mustermann-contrib (1.1.1)
       hansi (~> 0.2.0)
       mustermann (= 1.1.1)
+    openapi3_parser (0.8.2)
+      commonmarker (~> 0.17)
+      psych (~> 3.1)
     path (2.0.1)
-    public_suffix (4.0.5)
+    psych (3.2.1)
+    public_suffix (4.0.6)
     rack (2.2.3)
-    rack-protection (2.0.8.1)
+    rack-protection (2.1.0)
       rack
     rack-robustness (1.1.0)
     rack-test (0.6.3)
       rack (>= 1.0)
     rake (12.3.3)
-    rspec (3.9.0)
-      rspec-core (~> 3.9.0)
-      rspec-expectations (~> 3.9.0)
-      rspec-mocks (~> 3.9.0)
-    rspec-core (3.9.2)
-      rspec-support (~> 3.9.3)
-    rspec-expectations (3.9.2)
+    rspec (3.10.0)
+      rspec-core (~> 3.10.0)
+      rspec-expectations (~> 3.10.0)
+      rspec-mocks (~> 3.10.0)
+    rspec-core (3.10.0)
+      rspec-support (~> 3.10.0)
+    rspec-expectations (3.10.0)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.9.0)
-    rspec-mocks (3.9.1)
+      rspec-support (~> 3.10.0)
+    rspec-mocks (3.10.0)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.9.0)
-    rspec-support (3.9.3)
+      rspec-support (~> 3.10.0)
+    rspec-support (3.10.0)
     rspec_junit_formatter (0.4.1)
       rspec-core (>= 2, < 4, != 2.12.0)
+    ruby-enum (0.8.0)
+      i18n
     ruby2_keywords (0.0.2)
-    sinatra (2.0.8.1)
+    sinatra (2.1.0)
       mustermann (~> 1.0)
-      rack (~> 2.0)
-      rack-protection (= 2.0.8.1)
+      rack (~> 2.2)
+      rack-protection (= 2.1.0)
       tilt (~> 2.0)
     tilt (2.0.10)
     unf (0.1.4)
@@ -80,6 +94,7 @@ GEM
 
 PLATFORMS
   ruby
+  x86_64-darwin-15
 
 DEPENDENCIES
   rake (~> 12)
@@ -87,4 +102,4 @@ DEPENDENCIES
   webspicy!
 
 BUNDLED WITH
-   2.0.2
+   2.2.1

data/examples/restful/Rakefile

@@ -1,6 +1,5 @@
 $LOAD_PATH.unshift File.expand_path("..", __FILE__)
 require 'app'
-require 'webspicy'
 
 desc "Checks all .yml definition files"
 task :check do

data/examples/restful/app.rb

@@ -1,3 +1,4 @@
+require 'webspicy'
 require 'sinatra'
 require 'json'
 require 'path'
@@ -65,6 +66,7 @@ post '/reset' do
 end
 
 get '/todo/' do
+  raise unless request.env['HTTP_ACCEPT'] == 'application/json'
   content_type :json
   settings.todolist.to_json
 end
@@ -94,6 +96,7 @@ post '/todo/', :auth => :user do
 end
 
 get '/todo/:id' do |id|
+  raise unless request.env['HTTP_ACCEPT'] == 'application/json'
   content_type :json
   todo = settings.todolist.find{|todo| todo[:id] == Integer(id) }
   if todo.nil?
@@ -111,7 +114,7 @@ patch '/todo/:id', :auth => :user do |id|
     status 404
     {error: "No such todo"}.to_json
   else
-    patch = SCHEMA["TodoPatch"].dress(loaded_body)
+    patch = SCHEMA["TodoPatch"].dress(loaded_body.merge("id" => Integer(id)))
     patched = todo.merge(patch)
     settings.todolist = settings.todolist.reject{|todo| todo[:id] == Integer(id) }
     settings.todolist << patched

data/examples/restful/webspicy/config.rb

@@ -4,6 +4,14 @@ def webspicy_config(&bl)
     c.precondition MustBeAuthenticated
     c.precondition MustBeAnAdmin
 
+    c.precondition Webspicy::Specification::Precondition::GlobalRequestHeaders.new({
+      'Accept' => 'application/json'
+    }){|service| service.method == "GET" }
+
+    c.precondition Webspicy::Specification::Precondition::RobustToInvalidInput.new
+
+    c.postcondition TodoRemoved
+
     c.instrument do |tc, client|
       role = tc.metadata[:role]
       tc.headers['Authorization'] = "Bearer #{role}" if role

data/examples/restful/webspicy/rack.rb

@@ -1,6 +1,6 @@
 require_relative 'config'
 webspicy_config do |c|
-  c.client = Webspicy::RackTestClient.for(::Sinatra::Application)
+  c.client = Webspicy::Tester::RackTestClient.for(::Sinatra::Application)
   c.before_each do |_,client|
     client.api.post "/reset"
   end

data/examples/restful/webspicy/real.rb

@@ -1,7 +1,7 @@
 require_relative 'config'
 webspicy_config do |c|
   c.host = "http://127.0.0.1:4567"
-  c.client = Webspicy::HttpClient
+  c.client = Webspicy::Tester::HttpClient
   c.before_each do |_,client|
     client.api.post "http://127.0.0.1:4567/reset"
   end

data/examples/restful/webspicy/schema.fio

@@ -1,4 +1,4 @@
-@import finitio/data
+@import webspicy/scalars
 
 Todo = {
   id          : Integer
@@ -6,7 +6,7 @@ Todo = {
 }
 
 TodoPatch = {
-  id          :? Integer
+  id          :  Integer
   description :? String
 }
 

data/examples/restful/webspicy/support/must_be_authenticated.rb

@@ -30,7 +30,7 @@ class MustBeAuthenticated
   def counterexample(description, role, expected, status = 401)
     YAML.load <<-YML.gsub(/^\s+[#][ ]/, "")
     # description: |-
-    #   (#{self.class.name}) #{description}
+    #   #{description} (#{self.class.name} PRE)
     # params:
     #   id: 1
     # dress_params:
@@ -41,7 +41,7 @@ class MustBeAuthenticated
     #   content_type: application/json
     #   status: #{status}
     # assert:
-    #   - "pathFD('', error: '#{expected}')" 
+    #   - "pathFD('', error: '#{expected}')"
     YML
   end
 

data/examples/restful/webspicy/support/todo_removed.rb

@@ -0,0 +1,18 @@
+class TodoRemoved
+  include Webspicy::Specification::Postcondition
+
+  def self.match(service, descr)
+    return TodoRemoved.new if descr =~ /The todo has been removed/
+  end
+
+  def check(invocation)
+    client = invocation.client
+    id = invocation.test_case.params['id']
+    url = "/todo/#{id}"
+    response = client.api.get(url, {}, {
+      "Accept" => "application/json"
+    })
+    return nil if response.status == 404
+    "Todo `#{id}` was not deleted, it has been found"
+  end
+end

data/examples/restful/webspicy/todo/deleteTodo.yml

@@ -15,13 +15,16 @@ services:
   preconditions:
     - Must be an admin
 
+  postconditions:
+    - The todo has been removed
+
   input_schema: |-
     {
       id: Integer
     }
 
   output_schema: |-
-    .
+    Any
 
   error_schema: |-
     ErrorSchema