webspicy 0.15.8 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e1d234226f15d51777e61c010557ed1691660613cd87ba11ce19844b2f7cc812
-  data.tar.gz: 40fcd4639ce847f7f69f3471ef9676c64e8e6be489388563d5abd51ca1bf39d7
+  metadata.gz: 63c15addd49634a62ba637d18cfee736f405dc0e144ec688e2fda6861764a09c
+  data.tar.gz: 903d3a0ca2f20a5dbac1fafc4a1a89981736d527cb69668a06b41acb5ef04696
 SHA512:
-  metadata.gz: c4d0ec9a0567955bac41c5ee887164e783799e2baf66966fabff404147376c29192a4b28090c7d816a20fd15bc7e470b20136e253b9b1a7d663ac57de4958d99
-  data.tar.gz: 4ddccde82357c81066108133b0e5b63ea8b5a0b714556b56b4d4743ba26aaaf17a9187633061dc2f721b88833cdfb536f795d8256aa158723c4fa5a255957144
+  metadata.gz: d54f59f003313e38dd37104e56038b6ede06c7339718498e6dda9730de97acdd0fae5499d45463ef790f78332eb0d337a3f068e3dd08bfec901edb8125a710c2
+  data.tar.gz: c43077a9023045856e726c0d2dbe39abc83abbd198df4a765758e6578664bcc6ec44ec7df75e164b436ae6a89cb6fe0ee89b4fc995afd3fe068bef2e5e484144

data/README.md

@@ -1,46 +1,98 @@
 # Webspicy
 
-A description, specification and test framework for web services seen as black
-box software operations.
+A specification and test framework for web services seen as black-box software
+operations. Webspicy yields a better test coverage for a smaller testing effort.
+
+See webspicy in action and make the tutorial on https://yourbackendisbroken.dev
+
+Have a look at `doc/*.md` for vocabulary and vision as well as `ROADMAP.md`.
 
 ## Features
 
-* Declarative description of RESTful web-services + their tests
+* Declarative specification of HTTP web services + their tests
+
+* Framework/language agnostic: `webspicy` is written in Ruby, but can be used
+  to test web services for backends written in any language / framework.
+
+* Black box testing: `webspicy` focuses on web services seen as blackboxes. It
+  has no knowledge of the implementation, and focuses on HTTP and input/output
+  data instead. Investing in such testing makes those tests more stable and
+  your API design better.
+
+* Formal and human-friendly data schema with strong data matching semantics,
+  thanks to [http://finitio.io](http://finitio.io)
+
+* Test instrumentation and generation, based on PRE & POST contracts.
 
-* Framework/language agnostic: Webspicy is written in Ruby, but can be used to
-  test web services in any language / framework.
+* Extra goodness for Rubyists: being written in ruby, `webspicy` also supports
+  testing Rack applications directly (through rack/test), which boosts the
+  test suite.
 
-* Black box testing: Webspicy focuses on web services seen as blackboxes. It has
-  no knowledge of the implementation, and focuses on HTTP and input/output data
-  instead. Investing in such testing makes those tests more stable and your API
-  better.
+* Extra goodies: when a specification is written, it can also be used for
+  mocking the API, generating an openapi file, etc.
 
-* Test instrumentation and generation, based on declarative PRE & POST
-  conditions.
+## Is this used on real-world cases?
 
-* Extra goodness for Rubyists: being written in ruby, Webspicy also supports
-  testing Rack applications directly (through rack/test)
+Yes, `webspicy` is currently used on a dozen production components. Our biggest specification has 324 specification files for thousands of tests, 35% of them being generated.
 
-## Getting started
+## Getting started with the commandline
 
-Please have a look at the example first. It contains a simple Sinatra application
-with GET and POST restful services tested with the framework. The Rakefile contains
-the necessary tasks to run those tests.
+To install webspicy on your developer computer, install ruby then:
 
-## Executing the tests using the command line
+```
+gem install webspicy
+```
 
-Please have a look at the examples first. Then:
+Then execute webspicy help to see the options.
 
 ```
 webspicy --help
-webspicy path/to/your/config.rb
 ```
 
-## Using the docker image
+## Using the docker image(s)
+
+If you just want to play with the commandline tool without having to
+install ruby & webspicy, you can use the docker image we provide for
+the commandline:
+
+```
+docker run enspirit/webspicy --help
+```
 
-A docker image running your tests is provided. It expects a volume with the
-tests to run put in `/home/app`.
+If you have a specification & test suite somewhere, an easy way to run the
+whole suite (or integrate it in your continuous integration pipeline) is
+to use our `:tester` docker image. Just mount your test suite as a volume
+in `/home/app` and you are good to go:
 
 ```
-docker run enspirit/webspicy -v path/to/tests:/home/app
+docker run -v path/to/tests:/formalspec enspirit/webspicy:tester
 ```
+
+If your plan is to test a backend that runs on your own machine (vs.
+publicly on the Internet or inside the default docker network),
+you will need to add some networking option, as shown below. Please
+refer to Docker documentation.
+
+```
+docker run -v path/to/tests:/formalspec --network=host enspirit/webspicy:tester
+```
+
+## Contributing
+
+Please use github issues for questions and bugs, and pull requests for
+submitting improvement proposals and new features.
+
+## Contributors
+
+* Bernard Lambeau (blambeau)
+* Louis Lambeau (llambeau)
+* Yoann Guyot (ygu)
+* Felix Holmgren (@felixyz)
+
+Enspirit (https://enspirit.be) and Klaro App (https://klaro.cards) are
+both actively using, contributing and funding work on this library.
+Please contact Bernard Lambeau for any question.
+
+## Licence
+
+Webspicy is distributed under a MIT Licence, by Enspirit SRL.

data/bin/webspicy

@@ -1,15 +1,25 @@
 #!/usr/bin/env ruby
-#/ Document and test RESTful webservices
-#/ (c) Enspirit SPRL. Distributed under MIT licence.
-#/ Usage: webspicy [options] path/to/config.rb
+#/
+#/ Document & test web services as black-box operations.
+#/ (c) Enspirit SRL. Distributed under MIT licence.
+#/
+#/ Usage: webspicy [options] URL                 # run default test againt url
+#/ Usage: webspicy [options] path/to/config.rb   # run whole test suite
+#/ Usage: webspicy [options] path/to/spec.yml    # run a single test
+#/
+#/ Options:
+#/ -h, --help       Show this help message
+#/ -v, --version    Show webspicy version
+#/     --debug      Same as LOG_LEVEL=DEBUG, takes precedence
 #/
 #/ Recognized env vars:
-#/ - ROBUST=no                    do not execute counterexamples
+#/ - ROBUST=no|only|generated     execute no counterexamples (only them, only generated ones)
 #/ - LOG_LEVEL=INFO|DEBUG|...     set log level to the appropriate level
 #/ - RESOURCE=...                 restrict to .yml test files matching the pattern
-#/ - METHOD=GET|POST|DELETE...    execute only tests matching the verb
+#/ - METHOD=GET|POST|DELETE...    execute only tests matching the HTTP verb
 #/ - TAG=...                      execute only tests matching the given tag
 #/ - FAILFAST=yes|no              stop executing tests on first failure
+#/
 require 'webspicy'
 require 'webspicy/tester'
 require 'optparse'
@@ -20,19 +30,25 @@ def showhelp
 end
 
 ARGV.options do |opts|
-  opts.on_tail("-h", "--help"){ showhelp }
+  opts.on_tail("-h", "--help"){
+    showhelp
+  }
+  opts.on("-v", "--version"){
+    puts "webspicy v#{Webspicy::VERSION}, (c) Enspirit SRL"
+    exit(0)
+  }
+  opts.on("--debug") {
+    ENV['LOG_LEVEL'] = 'DEBUG'
+    Webspicy::LOGGER.level = Logger.const_get(ENV['LOG_LEVEL'])
+  }
   opts.parse!
 end
 
 if ARGV.size != 1
   showhelp
   exit(-1)
-elsif (p = Path(ARGV.first)).exists?
-  config = Webspicy::Configuration.dress(p)
-  res = Webspicy::Tester.new(config).call
-  abort("Some tests failed") unless res == 0
-else
-  puts "No such file or directory `#{ARGV.first}`"
-  showhelp
-  exit(-1)
 end
+
+config = Webspicy::Configuration.dress(ARGV[0])
+res = Webspicy::Tester.new(config).call
+abort("Some tests failed") unless res == 0

data/examples/restful/Gemfile.lock

@@ -1,15 +1,17 @@
 PATH
   remote: ../..
   specs:
-    webspicy (0.15.8)
-      finitio (>= 0.8.0)
+    webspicy (0.17.0)
+      colorize (~> 0.8.1)
+      finitio (~> 0.9.0)
       http (>= 2)
       mustermann (~> 1.0)
       mustermann-contrib
-      path (>= 1.3)
+      openapi3_parser (~> 0.8.2)
+      path (~> 2.0)
       rack-robustness (~> 1.1, >= 1.1.0)
       rack-test (~> 0.6.3)
-      rspec (~> 3.7)
+      rspec (~> 3.10)
       rspec_junit_formatter (~> 0.4.1)
 
 GEM
@@ -18,14 +20,18 @@ GEM
     addressable (2.7.0)
       public_suffix (>= 2.0.2, < 5.0)
     citrus (3.0.2)
+    colorize (0.8.1)
+    commonmarker (0.21.0)
+      ruby-enum (~> 0.5)
+    concurrent-ruby (1.1.7)
     diff-lcs (1.4.4)
     domain_name (0.5.20190701)
       unf (>= 0.0.5, < 1.0.0)
-    ffi (1.13.1)
+    ffi (1.14.2)
     ffi-compiler (1.0.1)
       ffi (>= 1.0.0)
       rake
-    finitio (0.8.0)
+    finitio (0.9.1)
       citrus (>= 2.4, < 4.0)
     hansi (0.2.0)
     http (4.4.1)
@@ -36,42 +42,50 @@ GEM
     http-cookie (1.0.3)
       domain_name (~> 0.5)
     http-form_data (2.3.0)
-    http-parser (1.2.1)
+    http-parser (1.2.3)
       ffi-compiler (>= 1.0, < 2.0)
+    i18n (1.8.7)
+      concurrent-ruby (~> 1.0)
     mustermann (1.1.1)
       ruby2_keywords (~> 0.0.1)
     mustermann-contrib (1.1.1)
       hansi (~> 0.2.0)
       mustermann (= 1.1.1)
+    openapi3_parser (0.8.2)
+      commonmarker (~> 0.17)
+      psych (~> 3.1)
     path (2.0.1)
-    public_suffix (4.0.5)
+    psych (3.3.0)
+    public_suffix (4.0.6)
     rack (2.2.3)
-    rack-protection (2.0.8.1)
+    rack-protection (2.1.0)
       rack
     rack-robustness (1.1.0)
     rack-test (0.6.3)
       rack (>= 1.0)
     rake (12.3.3)
-    rspec (3.9.0)
-      rspec-core (~> 3.9.0)
-      rspec-expectations (~> 3.9.0)
-      rspec-mocks (~> 3.9.0)
-    rspec-core (3.9.2)
-      rspec-support (~> 3.9.3)
-    rspec-expectations (3.9.2)
+    rspec (3.10.0)
+      rspec-core (~> 3.10.0)
+      rspec-expectations (~> 3.10.0)
+      rspec-mocks (~> 3.10.0)
+    rspec-core (3.10.1)
+      rspec-support (~> 3.10.0)
+    rspec-expectations (3.10.1)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.9.0)
-    rspec-mocks (3.9.1)
+      rspec-support (~> 3.10.0)
+    rspec-mocks (3.10.1)
       diff-lcs (>= 1.2.0, < 2.0)
-      rspec-support (~> 3.9.0)
-    rspec-support (3.9.3)
+      rspec-support (~> 3.10.0)
+    rspec-support (3.10.1)
     rspec_junit_formatter (0.4.1)
       rspec-core (>= 2, < 4, != 2.12.0)
+    ruby-enum (0.8.0)
+      i18n
     ruby2_keywords (0.0.2)
-    sinatra (2.0.8.1)
+    sinatra (2.1.0)
       mustermann (~> 1.0)
-      rack (~> 2.0)
-      rack-protection (= 2.0.8.1)
+      rack (~> 2.2)
+      rack-protection (= 2.1.0)
       tilt (~> 2.0)
     tilt (2.0.10)
     unf (0.1.4)
@@ -80,6 +94,8 @@ GEM
 
 PLATFORMS
   ruby
+  x86_64-darwin-15
+  x86_64-darwin-18
 
 DEPENDENCIES
   rake (~> 12)
@@ -87,4 +103,4 @@ DEPENDENCIES
   webspicy!
 
 BUNDLED WITH
-   2.0.2
+   2.2.2

data/examples/restful/Rakefile

@@ -1,6 +1,5 @@
 $LOAD_PATH.unshift File.expand_path("..", __FILE__)
 require 'app'
-require 'webspicy'
 
 desc "Checks all .yml definition files"
 task :check do

data/examples/restful/app.rb

@@ -1,3 +1,4 @@
+require 'webspicy'
 require 'sinatra'
 require 'json'
 require 'path'
@@ -65,6 +66,7 @@ post '/reset' do
 end
 
 get '/todo/' do
+  raise unless request.env['HTTP_ACCEPT'] == 'application/json'
   content_type :json
   settings.todolist.to_json
 end
@@ -94,6 +96,7 @@ post '/todo/', :auth => :user do
 end
 
 get '/todo/:id' do |id|
+  raise unless request.env['HTTP_ACCEPT'] == 'application/json'
   content_type :json
   todo = settings.todolist.find{|todo| todo[:id] == Integer(id) }
   if todo.nil?
@@ -111,7 +114,7 @@ patch '/todo/:id', :auth => :user do |id|
     status 404
     {error: "No such todo"}.to_json
   else
-    patch = SCHEMA["TodoPatch"].dress(loaded_body)
+    patch = SCHEMA["TodoPatch"].dress(loaded_body.merge("id" => Integer(id)))
     patched = todo.merge(patch)
     settings.todolist = settings.todolist.reject{|todo| todo[:id] == Integer(id) }
     settings.todolist << patched

data/examples/restful/webspicy/config.rb

@@ -4,6 +4,15 @@ def webspicy_config(&bl)
     c.precondition MustBeAuthenticated
     c.precondition MustBeAnAdmin
 
+    c.precondition Webspicy::Specification::Precondition::GlobalRequestHeaders.new({
+      'Accept' => 'application/json'
+    }){|service| service.method == "GET" }
+
+    c.precondition Webspicy::Specification::Precondition::RobustToInvalidInput.new
+
+    c.postcondition TodoRemoved
+    c.errcondition  TodoNotRemoved
+
     c.instrument do |tc, client|
       role = tc.metadata[:role]
       tc.headers['Authorization'] = "Bearer #{role}" if role

data/examples/restful/webspicy/{todo/deleteTodo.yml → formaldef/todo/_one/delete.yml}

@@ -15,13 +15,19 @@ services:
   preconditions:
     - Must be an admin
 
+  postconditions:
+    - The todo has been removed
+
+  errconditions:
+    - If it existed, the todo has not been removed
+
   input_schema: |-
     {
       id: Integer
     }
 
   output_schema: |-
-    .
+    Any
 
   error_schema: |-
     ErrorSchema

data/examples/restful/webspicy/formaldef/todo/_one/get.simpler.yml

@@ -0,0 +1,46 @@
+---
+url: |-
+  /todo/{id}
+
+method: |-
+  GET
+
+description: |-
+  Returns a single todo item
+
+input_schema: |-
+  {
+    id: Integer
+  }
+
+output_schema: |-
+  Todo
+
+error_schema: |-
+  ErrorSchema
+
+examples:
+
+  - description: |-
+      when requested on an existing TODO
+    params:
+      id: 1
+    expected:
+      content_type: application/json
+      status: 200
+    assert:
+      - "pathFD('', id: 1)"
+      - "match('description', /Refactor/)"
+      - "notMatch('description', /Foo/)"
+
+counterexamples:
+
+  - description: |-
+      when requested on an unexisting TODO
+    params:
+      id: 999254654
+    expected:
+      content_type: application/json
+      status: 404
+    assert:
+      - "pathFD('', error: 'No such todo')"