tla-sbuilder 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: c7c59bc7c8ccd8cc4c991d487f793208c1234bab
+  data.tar.gz: 564a18b8c901b8714dfb32c7c70d0a78b5a8f7ac
+SHA512:
+  metadata.gz: 208032b65aff58ad9a32ae7f4197da74b24979defcf30886bcbcd32974571b34da94c80f027dddea217bfea2a7d71c254aa8b93b06f7f94d80433960818c90dd
+  data.tar.gz: 667f6dd2647f7e60cb1bfde14f28d55703c57b1b16f3dc33e2fbf1ca7c61d320feb9d51ea89856d544ba48da6f295f2e35b21547e20d9beed63c11ff9985007a

data/README.md

@@ -0,0 +1,328 @@
+<link href="../site.css" rel="stylesheet"></link>
+[Up](../index.php) [Readme](README.html) [Releases](RELEASES.html) [Todo](TODO.html)
+
+
+# Sbuilder - A Specification Builder for TLA+ Tools - $Release:0.1.0$
+
+A tool to generate runnable specification models in
+[TLA+ language](http://research.microsoft.com/en-us/um/people/lamport/tla/book.html)
+for [target systems](#TARGET-SYSTEM). Specification model can be
+verified using
+[TLA+ Tools](http://research.microsoft.com/en-us/um/people/lamport/tla/tools.html),
+and parts of it can be presented as
+[implementation blueprints](#BLUEPRINT) to developers.
+
+See [live documentation](features.html) for more details.
+
+## <a id="TARGET-SYSTEM">Target Systems</a>
+
+Sbuilder supports modeling typical business IT systems, where a system
+architecture identifies
+
+* **interfaces**: entry point providing access to services within the
+  system. **Application interface** is the public entry point, and
+  **infrastructure interface** is the internal entry point allowing
+  aggregation of (application) services from smaller units.
+  
+* **services**: unit of functionality accessible over
+  interfaces. **Infrastructure service** is not directly accessible over
+  _application interface_, as opposed to **application services**
+  
+* **database**: persistent data store, which services may access, and
+  modify.
+
+## <a id="ELEMENTS-OF">Runnable Specification Code</a>
+
+Sbuilder combines three main elements to create runnable TLA+
+specification code:
+
+1. **Interface specification**: Sbuilder accepts
+  [RESTfull](https://en.wikipedia.org/wiki/Representational_state_transfer)
+  interface api specification expressed in
+  [Swagger](http://swagger.io/)
+  [v2.0](http://swagger.io/specification/) format.
+  
+2. **Specification extension**: Sbuilder tangles manually crafted code
+   snippets from Sbuilder *code repository* with the code generated
+   using *inteterface specifications*. The manual code snippets use
+   TLA+ language to model behavior and correctness criteria of the
+   target system.
+  
+3. **Setups**: To complete runnable code, Sbuilder uses setups. A
+   setup is a sequence of interface operations, and their input
+   parameters.  In addition, a setup may define domain cardinalities,
+   or domain values.  The tools allows defining multiple setups, each
+   setup resulting to a complete, runnable TLA+ code, allowing system
+   model to be run in various environment context
+
+Sbuilder uses following categories to organize *specification
+snippets* in its **code repository**:
+
+* **state**: TLA+ variables modeling *database*
+
+* **operator**: TLA+ language constructs, which enable "to define all
+   the data structures and operations that occur in the specification"
+
+* **transaction**: pseudo code implementation of common behavior in
+   modifying system state
+
+* **service**: pseudo code implementation of target system
+ *application services* using PlusCal procedures in TLA+ language
+
+* **infrastructure**: pseudo code implementation of target system
+  *infrastructure services* using PlusCal procedures in TLA+ language
+
+* **interface**: implementation of *application interface* extension
+    points using PlusCal macros in TLA+ language
+
+* **correctness**: operator definitions, and INVARIANT directives in
+  TLA+ language specifying properties, which must be not violated in
+  specification executions
+
+* **possibility**: optional operator definitions to increase
+  confidence in correctness invariants
+
+* **assumption**: operator definitions, and ASSUME directive in TLA+
+  language specifying properties, which must be not violated in the
+  specification code
+
+In order to support IT system application development, as depicted in
+section [modeling pipeline](#MODELING-PIPELINE) below, Sbuilder
+can generate <a id="BLUEPRINT">**implementation
+blueprints**</a>. *Implementation blueprints* are html-pages including
+code snippets from Sbuilder *code repository*, or code generated
+based on *interface specifications*.
+
+
+## <a id="MODELING-PIPELINE">Modeling Pipeline</a>
+
+The picture below gives an overview, how Sbuilder -tool can be
+deployed in an "mainstream" software development process.
+
+The objective for creating a specification model is to get better
+understanding, to get the design right, and to write better code for
+the system being built - and thereby to justify the extra effort
+needed in modeling.
+
+
+![process](pics/process.svg)
+
+We assume that services are currently implemented based on informal
+service specification documents written in English. We also assume
+that service interface is available in a machine readable format.
+
+In the picture, process enhancements are highlighted in a box labeled
+"Pseudo code modeling". The output of the process enhancement are
+implementation blueprints, which are made available for developers at
+the side of current service implementation documentation. An
+implementation blueprint for a service contains pseudo code snippets
+extracted from specification model, relevant for implementing a
+particular service.
+
+Pseudo code modeling includes following tasks:
+
+- **accept API specification input**: Machine readable interface
+  specification is translated to TLA+ language.
+
+- **create domain model**: Interface parameters are mapped to
+  domains. In this task, we define domain resolvers in Sbuilder -tool,
+  and add TLA+ operators, and assumptions to verify that the resulting
+  domain model conforms to service specification. In practice,
+  creating a formal domain model results in "Gaining better
+  understanding" of the informal specification.
+
+- **specify correctness**: In the spirit of Test Driven Development,
+  specifying correctness precedes specifying behavior. In this task,
+  we interpret test cases, (hopefully) presented in the informal
+  service specification, as TLA+ language operators and invariants.
+
+- **model behavior**: Modeling behavior includes creating pseudo code
+  service implementation in TLA+ language. In this task, when we have
+  formal correctness criteria expressed in TLA+, running model checker
+  points out traces leading to violations any of the criteria, and
+  helps us in "Getting design right".
+
+- **create implementation blueprint**: In the final task, modeler
+  defines web pages for implementation blueprints. A blueprint
+  collects relevant TLA+ snippets in Sbuilder code repository,
+  together with any other useful documentation, to help in
+  implementing a specific service. Web pages are made available for
+  service implementation to help developers to "Write better
+  code". Competent developers may understand TLA+ language, but, like
+  in software engineering generally, good comments in pseudo code
+  snippets will make blueprints more approachable.
+
+
+The picture above defines following roles:
+
+* **architect**: expert role specifying services, and defining
+  interfaces 
+
+* **modeler**: expert role crafting specification extensions, and
+using Sbuilder, and TLA+ tools to generate specification code, and
+to verify its correctness
+
+* **developer**: expert role implementing IT system interface and
+service according to the service specification, interface definition,
+and Sbuilder blueprint.
+
+
+## <a id="USAGE">Usage</a>
+
+### Pre-requisites
+
+`Sbuilder` -tool requires Ruby 2.
+
+To run the specification model created by Sbuilder -tool, you must
+have Java and
+[TLA+ Tools](http://research.microsoft.com/en-us/um/people/lamport/tla/tools.html)
+installed.
+
+
+### Installation
+
+To install `Sbuilder` Gem, create a `Gemfile` -file with the
+content
+
+```
+source "https://rubygems.org"
+gem 'tla-sbuilder'
+```
+
+and run
+
+	bundle install
+
+TLA+ Tools jar can be obtained from
+[download page](https://tla.msr-inria.inria.fr/tlatoolbox/dist), see
+[TLA+ Tools](http://research.microsoft.com/en-us/um/people/lamport/tla/tools.html)
+for more instructions on installation.
+
+### To Use Pet Store Example Application
+
+Create default directories used by `sbuilder` in current working directory
+
+	bundle exec sbuilder.rb init
+
+Create example configuration
+
+	bundle exec sbuilder.rb example pet
+
+To list configuration files created for pet store example
+
+	ls cnf
+
+
+To list specification extensions in TLA-sbuilder code repository
+
+	ls src/pet
+
+
+To generate all setups in petstore example
+
+	bundle exec sbuilder.rb generate -t src/pet/
+
+To run model checking for setup `pet1` using TLA+tools jar in
+`~/java/tla/tla2tools.jar`
+
+    export CP=~/java/tla/tla2tools.jar
+    (cd gen/pet1/tla; java -cp $CP pcal.trans model)
+	(cd gen/pet1/tla; java -cp  $CP tlc2.TLC setup)
+
+and observe **Model checking completed. No error has been found.** to
+conclude that **correctness invariants** for the model are not
+violated.
+
+
+In setup `pet1`, it is **NOT possible** to reach a state where
+predicate
+
+	at_least_two_tags == Cardinality( v_tags ) > 1
+
+holds.  This is because setup steps in `cnf/extend_petstore_run1.yaml`
+do not call interface operation `/tags(post)`.
+
+To show that it is **NOT possible** to reach a state, where
+`at_least_two_tags` holds, run
+
+	(cd gen/pet1/tla; java -cp  $CP tlc2.TLC possible_tag_with_invalid_address)
+
+and observe **Model checking completed. No error has been found.**.
+
+However, steps for setup `pet3`, defined in
+`cnf/extend_petstore_run3.yaml`, do call `/tags(post)` twice, making
+it possible to reach a state where predicate `at_least_two_tags`
+holds.
+
+The **possibility** to reach this state can be demonstrated running
+
+    (cd gen/pet3/tla; java -cp $CP pcal.trans model)
+	(cd gen/pet3/tla; java -cp  $CP tlc2.TLC possible_tag_with_invalid_address)	
+
+and observing **Error: Evaluating invariant** `possible_tag_with_invalid_address` **failed.**
+
+
+### Create Own Model
+
+See [live documentation](features.html) for more details.
+
+Create default directories used by `sbuilder` in current working directory
+
+	bundle exec sbuilder.rb init
+
+
+Create extension templates for including *specification extensions* in
+default in TLA-sbuilder *code repository* into specification model.
+
+	bundle exec sbuilder.rb extend
+
+
+Remove `.example` from all files in `cnf` directory
+
+Generate example setups: 
+
+	bundle exec sbuilder.rb generate
+
+To run the model checking for setup `default` using TLA+tools jar in
+`~/java/tla/tla2tools.jar`
+
+    export CP=~/java/tla/tla2tools.jar
+    (cd gen/default/tla; java -cp $CP pcal.trans model)
+	(cd gen/default/tla; java -cp  $CP tlc2.TLC setup)
+
+
+## Manage State Space Explosion, and Allow Scaling
+
+[State space explosion](https://www.google.fi/search?q=state+space+explosion)
+is inherent problem in
+[model checking](https://en.wikipedia.org/wiki/Model_checking), the
+theory behind TLA+ tool.
+
+Sbuilder tries to avoid problems due to state space explosion by 
+
+* allowing controlling, how environment invokes interface operations,
+  and in effect, throttling number of states generated during model
+  checking.
+
+* allowing to fix input parameter bindings in an interface
+  operation. In cases, where domain sizes are not a problem, SBuilder
+  can allow TLA+ tool to explore all possible, or partial, input
+  parameter combinations during model checking.
+
+
+In TLA+tools, number of states/sec decreases as specification code
+volume increases (see a [benchmark](BENCHMARK.html) results for more
+details).
+
+SBuilder tries to avoid problems due to increase in specification code
+by offering run time option `--filter-src`, which parses TLA+ snippets
+and runs static call flow analysis to prune off unused snippets from
+specification model. Parser is in alpha phase, in case of error, use
+option `--filter-list` to continue after parser error, and to
+configure manually modules call flow analyzer would otherwise omit
+from the specification code.
+
+
+## License 
+
+MIT

data/VERSION

@@ -0,0 +1 @@
+0.1.0

data/bin/sbuilder.rb

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require_relative "../lib/cli/cli.rb"
+
+Sbuilder::Cli.start

data/lib/cli/cli-customer.rb

@@ -0,0 +1,420 @@
+module Sbuilder
+  
+  require_relative 'cli-text'
+
+  class CliCustomer
+
+    # ------------------------------------------------------------------
+    # Local constsants
+    # PETSTORE_URL="https://rawgit.com/swagger-api/swagger-spec/master/examples/v2.0/yaml/petstore-expanded.yaml"
+    # file names
+    FILE_RESOLVER_CUSTOMER="resolver_customer.yaml"
+    FILE_INTERFACE_CUSTOMER="interface_customer.yaml"
+
+    
+    FILE_EXTENSIONS_CUSTOMER_COMMON="extend_customer_doms.yaml"
+    FILE_EXTENSIONS_CUSTOMER_RUN1="extend_customer_run1.yaml"
+    FILE_EXTENSIONS_CUSTOMER_RUN2="extend_customer_run2.yaml"                
+    
+    SBUILDER_YAML_CUSTOMER= <<-EOS
+
+    # ------------------------------------------------------------------
+    #{Sbuilder::CliText::SBUILBER_INTERFACES}
+
+    interfaces:
+        -  type: swagger
+           file: #{FILE_INTERFACE_CUSTOMER}
+
+
+    # ------------------------------------------------------------------
+    #{Sbuilder::CliText::SBUILDER_RESOLVERS}
+    resolvers:
+        -  type: resolver_yaml
+           file: #{FILE_RESOLVER_CUSTOMER}
+
+    # ------------------------------------------------------------------
+    #{Sbuilder::CliText::SBUILER_SETUPS}
+    setups:
+        - setupDirectory: default
+
+        - setupDirectory: customer1
+          extensions:
+            -  type: default-yaml
+               url: cnf/#{FILE_EXTENSIONS_CUSTOMER_COMMON}
+            -  type: default-yaml
+               url: cnf/#{FILE_EXTENSIONS_CUSTOMER_RUN1}
+
+        - setupDirectory: customer2
+          extensions:
+            -  type: default-yaml
+               url: cnf/#{FILE_EXTENSIONS_CUSTOMER_COMMON}
+            -  type: default-yaml
+               url: cnf/#{FILE_EXTENSIONS_CUSTOMER_RUN2}
+
+    # ------------------------------------------------------------------
+    #{Sbuilder::CliText::SBUILDER_PREFERENCES}
+    preferences:
+          debug-output: true
+
+    # ------------------------------------------------------------------
+    #{Sbuilder::CliText::SBUILDER_GENERATE}
+
+
+    EOS
+
+    
+
+
+
+    EXTENSIONS_CUSTOMER_COMMON = <<-EOS
+    #{Sbuilder::CliText::EXTENSION_HEADER_DOM}
+    #
+    #
+    - 
+      domain-extension:
+           - domain: customer_id
+             cardinality: 2
+           - domain: customer_type
+             values:
+                  - business
+                  - consumer
+           - domain: customer_name
+             cardinality: 2
+           - domain: customer_street
+             cardinality: 1
+           - domain: customer_city
+             cardinality: 1
+
+    EOS
+
+    EXTENSIONS_CUSTOMER_RUN1 = <<-EOS
+    #{Sbuilder::CliText::EXTENSION_HEADER_SETUP}
+    #
+    # Extend domain resolved in #{FILE_RESOLVER_CUSTOMER} 
+    #
+    - step-extension:
+
+           - interface: /customers(post)
+             bindExact: true
+             inputs:
+                - input:
+                    customers:
+                      - 
+                        id: 1
+                        type: business
+                        _default: 1
+                        address:
+                          _default: 1
+                      - 
+                        id: 2
+                        _default: 1
+                        address:
+                          _default: Nil
+
+                - input:
+                    customers:
+                      - 
+                        id: 1
+                        type: consumer
+                        _default: 1
+                        address:
+                          _default: 1
+                      - 
+                        id: 2
+                        _default: 1
+                        address:
+                          _default: 1
+
+                - input:
+                    customers:
+                      - 
+                        id: 1
+                        type: business
+                        _default: 1
+                        address:
+                          _default: 1
+                      - 
+                        id: 2
+                        _default: 1
+                        address:
+                          _default: 1
+
+           # - interface: /customers(post)
+           #   bindExact: true
+           #   input:
+           #          customers:
+           #            - id: 1
+           #              _default: 1
+           #              address:
+           #                _default: Nil
+
+           # - interface: /customers(post)
+           #   bindExact: true
+           #   input:
+           #          customers:
+           #            - id: 1
+           #              _default: Nil
+           #              address:
+           #                _default: Nil
+           #            - id: 1
+           #              _default: Nil
+           #              address:
+           #                _default: 1
+
+
+           # - interface: /customers(post)
+           #   input:
+           #          # _default: 1
+           #          customers:
+           #            - id: 1
+           #              # name: 1
+           #            - id: 2
+           #              # name: 2
+           # #            - id: 3
+           # #              name: 1
+
+    EOS
+    
+    EXTENSIONS_CUSTOMER_RUN2 = <<-EOS
+    #{Sbuilder::CliText::EXTENSION_HEADER_SETUP}
+    #
+    #
+    - step-extension:
+
+           - interface: /customer(post)
+             input:
+                customer:
+                    # name: 2
+                    _default: 2
+                    address:
+                       _default: Nil
+                    #    street: 1
+                       # city: 1
+
+           # - interface: /customers(post)
+           #   input:
+           #          # _default: 1
+           #          customers:
+           #            - id: 1
+           #              # name: 1
+           #            - id: 2
+
+
+
+    EOS
+    
+    
+
+    RESOLVER_CUSTOMER = <<-EOS
+    #{Sbuilder::CliText::RESOLVER_HEADER}
+    #
+    # Example mapper for customer interface
+
+    -    Name: Customer
+         Matcher: Customer
+         Rules: 
+          - Matcher: name
+            Domain: customer_name
+
+          - Matcher: id
+            Domain: customer_id
+
+          - Matcher: type
+            Domain: customer_type
+
+    -    Name: Address
+         Matcher: Address
+         Rules: 
+          - Matcher: street
+            Domain: customer_street
+
+          - Matcher: city
+            Domain: customer_city
+
+
+    -    Name: Default
+         Rules: 
+          - Matcher: !ruby/regexp /.*/
+            Domain: default
+
+    -    Name: Customers
+         Matcher: Customers
+         Rules: 
+          - Ref: Customer
+
+
+
+    -    Name: customers_post
+         Matcher: /customers(post)
+         Rules: 
+             - Ref: Customer
+
+    -    Name: Error
+         Matcher: Error
+         Rules: 
+          - Matcher: code
+            Domain: error_codes
+          - Matcher: !ruby/regexp /.*/
+            Domain: default
+
+
+    -    Name: customer_get
+         Matcher: /customer(get)
+         Rules: 
+             - Ref: Customer
+             - Ref: Default
+
+    -    Name: customer_post
+         Matcher: /customer(post)
+         Rules: 
+             - Ref: Customer
+             - Ref: Default
+
+    -    Name: customers_get
+         Matcher: /customers(get)
+         Rules: 
+             - Matcher: names
+               Domain: customer_name
+             - Ref: Default
+
+
+    EOS
+
+    
+    INTERFACE_CUSTOMER_SWAGGER = <<-EOS
+    # Example YAML configuration file
+    # 
+    swagger: "2.0"
+    info:
+      version: 1.0.0
+      title: Sbuild demo customer
+      description: A simple Customer API
+      termsOfService: http://swagger.io/terms/
+      contact:
+        name: TLA Sbuilder team
+        email: foo@example.com
+        url: http://localhost
+      license:
+        name: MIT
+        url: http://github.com/gruntjs/grunt/blob/master/LICENSE-MIT
+    host: locahost
+    basePath: /api
+    schemes:
+      - http
+    consumes:
+      - application/json
+    produces:
+      - application/json
+    paths:
+      /customer:
+        get:
+          operationId: findCustomers by id
+          parameters:
+            - name: id
+              in: query
+              description: tags to filter by
+              required: false
+              type: array
+              collectionFormat: csv
+              items:
+                type: string
+            - name: limit
+              in: query
+              description: maximum number of results to return
+              required: false
+              type: integer
+              format: int32
+          responses:
+             200:
+                description: OK
+        post: 
+          operationId: postCustomer
+          parameters:
+            - name: customer
+              in: body
+              description: Customers to enter
+              required: true
+              schema:
+                  $ref: '#/definitions/Customer'
+          responses:
+             200:
+                description: OK
+      /customers:
+        post: 
+          operationId: postCustomers
+          parameters:
+            - name: customers
+              in: body
+              description: Customers to enter
+              required: true
+              schema:
+                  $ref: '#/definitions/Customers'
+          responses:
+             200:
+                description: OK
+        get:
+          operationId: findCustomers by tag
+          parameters:
+            - name: names
+              in: query
+              description: query by names
+              required: false
+              type: array
+              collectionFormat: csv
+              items:
+                type: string
+            - name: limit
+              in: query
+              description: maximum number of results to return
+              required: false
+              type: integer
+              format: int32
+          responses:
+            200:
+              description: customer response
+              schema:
+                type: array
+                items:
+                  $ref: '#/definitions/Customer'
+            default:
+              description: unexpected error
+              schema:
+                $ref: '#/definitions/Error'
+    definitions:
+           Address:
+             properties:
+               street:
+                type: string
+               city:
+                type: string
+           Customers:
+                type: array
+                allOf:
+                  - $ref: '#/definitions/Customer'
+           Customer:
+             properties:
+               id:
+                type: string
+               type:
+                type: string
+               name:
+                type: string
+               address:
+                  $ref: '#/definitions/Address'
+         
+           Error:
+             required:
+               - code
+               - message
+             properties:
+               code:
+                 type: integer
+                 format: int32
+               message:
+                 type: string
+    EOS
+
+    
+    
+  end
+end