mustrd 0.1.8__tar.gz → 0.2.1__tar.gz

{mustrd-0.1.8 → mustrd-0.2.1}/LICENSE

@@ -1,21 +1,21 @@
-MIT License
-
-Copyright (c) 2023 Semantic Partners Ltd
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+MIT License
+
+Copyright (c) 2023 Semantic Partners Ltd
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

{mustrd-0.1.8 → mustrd-0.2.1}/PKG-INFO

@@ -1,17 +1,18 @@
 Metadata-Version: 2.1
 Name: mustrd
-Version: 0.1.8
+Version: 0.2.1
 Summary: A Spec By Example framework for RDF and SPARQL, Inspired by Cucumber.
 Home-page: https://github.com/Semantic-partners/mustrd
 License: MIT
 Author: John Placek
 Author-email: john.placek@semanticpartners.com
-Requires-Python: ==3.11.7
+Requires-Python: >=3.11.7,<4.0.0
 Classifier: Framework :: Pytest
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Natural Language :: English
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Software Development :: Quality Assurance
 Classifier: Topic :: Software Development :: Testing
 Classifier: Topic :: Utilities
@@ -22,6 +23,7 @@ Requires-Dist: colorlog (>=6.7.0,<7.0.0)
 Requires-Dist: coverage (==7.4.3)
 Requires-Dist: flake8 (==7.0.0)
 Requires-Dist: multimethods-py (>=0.5.3,<0.6.0)
+Requires-Dist: numpy (>=1.26.0,<2.0.0)
 Requires-Dist: openpyxl (>=3.1.2,<4.0.0)
 Requires-Dist: pandas (>=1.5.2,<2.0.0)
 Requires-Dist: pyanzo (>=3.3.10,<4.0.0)

{mustrd-0.1.8 → mustrd-0.2.1}/README.adoc

@@ -1,58 +1,58 @@
-== Mustrd
-
-// tag::body[]
-
-image::https://github.com/Semantic-partners/mustrd/raw/python-coverage-comment-action-data/badge.svg[Coverage badge,link="https://github.com/Semantic-partners/mustrd/tree/python-coverage-comment-action-data"]
-
-=== Why?
-
-How do you know your SPARQL, whether it's in a pipeline, or a query, is doing what you intend? 
-
-As much as we love RDF and SPARQL and Semantic Tech in general, we found a small gap in tooling which would give us that certainty. 
-
-We missed the powerful testing frameworks that have evolved in imperative languages that help ensure you've written code that does what you think it should. 
-
-We wanted to be able to:
-
-* setup data scenarios and ensure queries worked as expected
-* setup edge cases for queries and ensure they still work
-* isolate small sparql enrichment / transformation steps and to know we're only INSERTing what we intend
-
-Enter MustRD. 
-
-=== What?
-
-MustRD is a Spec-By-Example ontology, with a reference python implementation, inspired by the likes of Cucumber. 
-
-It's designed to be triplestore/SPARQL engine agnostic (aren't open standards *wonderful*!). 
-
-=== What it is NOT
-MustRD is nothing to do with SHACL, or an alternative to it. In fact, we use SHACL for some of our features. 
-
-SHACL provides validation around data. 
-
-MustRD provides validation around data transformations. 
-
-=== How?
-You define your specs in ttl, or trig files. 
-We use the SBE approach of *Given*, *When*, *Then* to define starting dataset, an action, and a set of expectations. We build up a set of data. 
-Then, depending on whether your SPARQL is a CONSTRUCT, SELECT or a INSERT/DELETE, we run it, and compare results against a set of expectations (*Then*) that are defined in the same way as a *Given* .
-Alternatively, you could define your *Then*
-
-* as an explicit ASK, or
-* select; or 
-* in a higher-order expectation language like you will be used to in various platforms, a set of expectations.
-
-
-=== When?
-
-Soon. It's a work in progress, and we're building the things *we* need for the projects we work on at multiple clients, with multiple vendor stacks. 
-We already think it's useful, but it might not meet *your* needs, out of the box. 
-
-We invite you to try it, see where it doesn't fit, and raise an issue, or even better, a PR! If you need something custom, please check out our consultancy rates, and we might be able to prioritise a new feature for you.
-
-== Support
-We're a specialist consultancy in Semantic Tech, we're putting this out in case it's useful, but if you need more support, kindly contact our business team on info@semanticpartners.com
-
-// tag::body[]
-include::src/README.adoc[tags=body]
+== Mustrd
+
+// tag::body[]
+
+image::https://github.com/Semantic-partners/mustrd/raw/python-coverage-comment-action-data/badge.svg[Coverage badge,link="https://github.com/Semantic-partners/mustrd/tree/python-coverage-comment-action-data"]
+
+=== Why?
+
+How do you know your SPARQL, whether it's in a pipeline, or a query, is doing what you intend? 
+
+As much as we love RDF and SPARQL and Semantic Tech in general, we found a small gap in tooling which would give us that certainty. 
+
+We missed the powerful testing frameworks that have evolved in imperative languages that help ensure you've written code that does what you think it should. 
+
+We wanted to be able to:
+
+* setup data scenarios and ensure queries worked as expected
+* setup edge cases for queries and ensure they still work
+* isolate small sparql enrichment / transformation steps and to know we're only INSERTing what we intend
+
+Enter MustRD. 
+
+=== What?
+
+MustRD is a Spec-By-Example ontology, with a reference python implementation, inspired by the likes of Cucumber. 
+
+It's designed to be triplestore/SPARQL engine agnostic (aren't open standards *wonderful*!). 
+
+=== What it is NOT
+MustRD is nothing to do with SHACL, or an alternative to it. In fact, we use SHACL for some of our features. 
+
+SHACL provides validation around data. 
+
+MustRD provides validation around data transformations. 
+
+=== How?
+You define your specs in ttl, or trig files. 
+We use the SBE approach of *Given*, *When*, *Then* to define starting dataset, an action, and a set of expectations. We build up a set of data. 
+Then, depending on whether your SPARQL is a CONSTRUCT, SELECT or a INSERT/DELETE, we run it, and compare results against a set of expectations (*Then*) that are defined in the same way as a *Given* .
+Alternatively, you could define your *Then*
+
+* as an explicit ASK, or
+* select; or 
+* in a higher-order expectation language like you will be used to in various platforms, a set of expectations.
+
+
+=== When?
+
+Soon. It's a work in progress, and we're building the things *we* need for the projects we work on at multiple clients, with multiple vendor stacks. 
+We already think it's useful, but it might not meet *your* needs, out of the box. 
+
+We invite you to try it, see where it doesn't fit, and raise an issue, or even better, a PR! If you need something custom, please check out our consultancy rates, and we might be able to prioritise a new feature for you.
+
+== Support
+We're a specialist consultancy in Semantic Tech, we're putting this out in case it's useful, but if you need more support, kindly contact our business team on info@semanticpartners.com
+
+// tag::body[]
+include::src/README.adoc[tags=body]

{mustrd-0.1.8 → mustrd-0.2.1}/mustrd/README.adoc

@@ -1,201 +1,210 @@
-= Developer helper
-// tag::body[]
-
-== Try it out
-
-Ensure you have python3 installed, before you begin.
-To install the necessary dependencies, run the following command from the project root.
-
-`pip3 install  -r requirements.txt`
-
-Run the following command to execute the accompanying tests specifications.
-
-`python3 src/run.py -v -p "test/test-specs" -g "test/data" -w "test/data" -t "test/data"`
-
-You will see some warnings. Do not worry, some tests specifications are invalid and intentionally skipped.
-
-For a brief explanation of the meaning of these options use the help option.
-
-`python3 src/run.py --help`
-
-== Run the tests
-
-Run `pytest` from the project root.
-
-== Creating your own Test Specifications
-
-If you have got this far then you are probably ready to create your own specifications to test your application SPARQL queries. These will be executed against the default RDFLib triplestore unless you configure one or more alternatives. The instructions for this are included in <<Configuring external triplestores>> below.
-
-=== Givens
-These are used to specify the dataset against which the SPARQL statement will be run.
-They can be generated from external sources such as an existing graph, or a file or folder containing serialised RDF. It is also possible to specify the dataset as reified RDF directly in the test step. Currently tabular data sources such as csv files or TableDatasets are not supported.
-Multiple given statements can be supplied and data is combined into a single dataset for the test.
-
-* *InheritedDataset* - This is where no data is specified but the existing data in the target graph is retained rather than being replaced with a defined set. This can be used to chain tests together or to perform checks on application data.
-----
-    must:given [ a must:InheritedDataset ] ;
-----
-* *FileDataset* - The dataset is a local file containing serialised RDF. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX. The data is used to replace any existing content in the target graph for the test.
-----
-    must:given [ a must:FileDataset ;
-                 must:file "test/data/given.ttl" . ] ;
-----
-* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -g option on the command line.
-----
-    must:given [ a must:FolderDataset ;
-                 must:fileName "given.ttl" ] ;
-----
-* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements. e.g.
-----
-    must:given [ a must:StatementsDataset ;
-                 must:hasStatement [ a rdf:Statement ;
-                                     rdf:subject   test-data:sub ;
-                                     rdf:predicate test-data:pred ;
-                                     rdf:object    test-data:obj ; ] ; ] ;
-----
-* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-    must:given [ a must:AnzoGraphmartDataset ;
-                 must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
-                 must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
-    ] .
-----
-=== Whens
-These are the actual SPARQL queries that you wish to test. Queries can be supplied as a string directly in the test or as a file containing the query. Only single When statements are currently supported.
-Mustrd does not derive the query type from the actual query, so it is necessary to provide this in the specification. Supported query types are SelectSparql, ConstructSparql and UpdateSparql.
-
-* *TextSparqlSource* - The SPARQL query is included in the test as a (multiline) string value for the property queryText.
-e.g.
-----
-    must:when  [ a must:TextSparqlSource ;
-                 must:queryText "SELECT ?s ?p ?o WHERE { ?s ?p ?o }" ;
-                 must:queryType must:SelectSparql ] ;
-----
-
-* *FileSparqlSource* - The SPARQL query is contained in a local file.
-e.g.
-----
-    must:when  [ a must:FileSparqlSource  ;
-                 must:file "test/data/construct.rq" ;
-                 must:queryType must:ConstructSparql  ; ] ;
-----
-* *FolderSparqlSource* - Similar to the file SPARQL source except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -w option on the command line.
-----
-    must:when  [ a must:FolderSparqlSource ;
-                 must:fileName "construct.rq" ;
-                 must:queryType must:ConstructSparql  ; ] ;
-----
-* *AnzoQueryBuilderDataset* - The query is saved in the Query Builder of an Anzo instance and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-   must:when  [ a must:AnzoQueryBuilderDataset ;
-                must:queryFolder "Mustrd";
-                must:queryName "mustrd-construct" ;
-                must:queryType must:ConstructSparql
-    ];
-----
-=== Thens
-Then clauses are used to specify the expected result dataset for the test. These datasets can be specified in the same way as <<Givens>> except that an extended set of dataset types is supported. For the tabular results of SELECT queries TabularDatasets are required and again can be in file format such as CSV, or an inline table within the specification.
-* *FileDataset* - The dataset is a local file containing serialised RDF or tabular data. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX, as well as tabular formats (.csv, .xls, .xlsx).
-----
-    must:then  [ a must:FileDataset ;
-                 must:file "test/data/thenSuccess.xlsx" ] .
-----
-----
-    must:then  [ a must:FileDataset ;
-                 must:file "test/data/thenSuccess.nt" ] .
-----
-* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -t option on the command line.
-----
-    must:then [ a must:FolderDataset ;
-                 must:fileName "then.ttl" ] ;
-----
-* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements e.g.
-----
-    must:then [ a must:StatementsDataset ;
-                 must:hasStatement [ a rdf:Statement ;
-                                     rdf:subject   test-data:sub ;
-                                     rdf:predicate test-data:pred ;
-                                     rdf:object    test-data:obj ; ] ; ] ;
-----
-* *TableDataset* - The contents of the table defined in RDF syntax within the specification.
-E.g. a table dataset consisting of a single row and three columns.
-----
-    must:then  [ a must:TableDataset ;
-                   must:hasRow [ must:hasBinding[
-                        must:variable "s" ;
-                        must:boundValue  test-data:sub ; ],
-                      [ must:variable "p" ;
-                        must:boundValue  test-data:pred ; ],
-                      [ must:variable "o" ;
-                        must:boundValue  test-data:obj ; ] ;
-               ] ; ] .
-----
-* *OrderedTableDataset* -  This is an extension of the TableDataset which allows the row order of the dataset to be specified using the SHACL order property to support the ORDER BY clause in SPARQL SELECT queries
-E.g. A table dataset consisting of two ordered rows and three columns.
-----
-    must:then  [ a must:OrderedTableDataset ;
-                 must:hasRow [ sh:order 1 ;
-                             must:hasBinding[ must:variable "s" ;
-                                        must:boundValue  test-data:sub1 ; ],
-                                      [ must:variable "p" ;
-                                        must:boundValue  test-data:pred1 ; ],
-                                      [ must:variable "o" ;
-                                        must:boundValue  test-data:obj1 ; ] ; ] ,
-                            [ sh:order 2 ;
-                             must:hasBinding[ must:variable "s" ;
-                                        must:boundValue  test-data:sub2 ; ],
-                                      [ must:variable "p" ;
-                                        must:boundValue  test-data:pred2 ; ],
-                                      [ must:variable "o" ;
-                                        must:boundValue  test-data:obj2 ; ] ; ] ;
-               ] .
-----
-* *EmptyTable* - This is used to indicate that we are expecting an empty result from a SPARQL SELECT query.
-----
-    must:then  [ a must:EmptyTable ] .
-----
-* *EmptyGraph* - Similar to EmptyTable but used to indicate that we are expecting an empty graph as a result from a SPARQL query.
-----
-    must:then  [ a must:EmptyGraph ] .
-----
-* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-    must:then [ a must:AnzoGraphmartDataset ;
-                must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
-                must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
-        ] .
-----
-== Configuring external triplestores
-The configuration file for external triplestores can be located outside of the project root as it is specified as an argument to the mustard module or as the -c option on the commandline when running run.py.
-
-It is anticipated that the external triplestore is running as mustrd is not configured to start them.
-
-Currently, the supported external triplestores are GraphDB and Anzo.
-
-The configuration file should be serialised RDF. An example in Turtle format is included below for GraphDB. For Anzo the *must:repository* value is replaced with a *must:gqeURI*.
-----
-@prefix must:      <https://mustrd.com/model/> .
-must:GraphDbConfig1  a must:GraphDbConfig ;
-        must:url "http://localhost";
-        must:port "7200";
-        must:inputGraph "http://localhost:7200/test-graph" ;
-        must:repository "mustrd" .
-----
-To avoid versioning secrets when you want to version triplestore configuration (for example in case you want to run mustrd in CI), you have to configure user/password in a different file.
-This file must be named as the triple store configuration file, but with "_secrets" just before the extension. For example triplestores.ttl -> triplestores_secrets.ttl
-Subjects in the two files must match, no need to redefine the type, for example:
-----
-@prefix must:      <https://mustrd.com/model/> .
-must:GraphDbConfig1  must:username 'test' ;
-              must:password 'test' .
-----
-
-== Additional Notes for Developers
-Mustrd remains very much under development. It is anticipated that additional functionality and triplestore support will be added over time. The project uses https://python-poetry.org/docs/[Poetry] to manage dependencies so it will be necessary to have this installed to contribute towards the project. The link contains instructions on how to install and use this.
-As the project is actually built from the requirements.txt file at the project root, it is necessary to export dependencies from poetry to this file before committing and pushing changes to the repository, using the following command.
-
-`poetry export -f requirements.txt --without-hashes > requirements.txt`
-
-
-
-// end::body[]
+= Developer helper
+// tag::body[]
+
+== Try it out
+
+Ensure you have python3 installed, before you begin.
+To install the necessary dependencies, run the following command from the project root.
+
+`pip3 install  -r requirements.txt`
+
+Run the following command to execute the accompanying tests specifications.
+
+`python3 src/run.py -v -p "test/test-specs" -g "test/data" -w "test/data" -t "test/data"`
+
+You will see some warnings. Do not worry, some tests specifications are invalid and intentionally skipped.
+
+For a brief explanation of the meaning of these options use the help option.
+
+`python3 src/run.py --help`
+
+== Run the tests
+
+Run `pytest` from the project root.
+
+== Creating your own Test Specifications
+
+If you have got this far then you are probably ready to create your own specifications to test your application SPARQL queries. These will be executed against the default RDFLib triplestore unless you configure one or more alternatives. The instructions for this are included in <<Configuring external triplestores>> below.
+
+=== Paths
+All paths are consired relative. That way mustrd tests can be versionned and shared easily.
+To get absolute path from relative path in a spec file, we prefix it with the first existing result in:
+1) Path where the spec is located
+2) spec_path defined in mustrd test configuration files or cmd line argument
+3) data_path defined in mustrd test configuration files or cmd line argument
+4) Mustrd folder: In case of default resources packaged with mustrd source (will be in venv when mustrd is called as library)
+We intentionally use the same method to build paths in all spec components to avoid confusion.
+
+=== Givens
+These are used to specify the dataset against which the SPARQL statement will be run.
+They can be generated from external sources such as an existing graph, or a file or folder containing serialised RDF. It is also possible to specify the dataset as reified RDF directly in the test step. Currently tabular data sources such as csv files or TableDatasets are not supported.
+Multiple given statements can be supplied and data is combined into a single dataset for the test.
+
+* *InheritedDataset* - This is where no data is specified but the existing data in the target graph is retained rather than being replaced with a defined set. This can be used to chain tests together or to perform checks on application data.
+----
+    must:given [ a must:InheritedDataset ] ;
+----
+* *FileDataset* - The dataset is a local file containing serialised RDF. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX. The data is used to replace any existing content in the target graph for the test.
+----
+    must:given [ a must:FileDataset ;
+                 must:file "test/data/given.ttl" . ] ;
+----
+* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -g option on the command line.
+----
+    must:given [ a must:FolderDataset ;
+                 must:fileName "given.ttl" ] ;
+----
+* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements. e.g.
+----
+    must:given [ a must:StatementsDataset ;
+                 must:hasStatement [ a rdf:Statement ;
+                                     rdf:subject   test-data:sub ;
+                                     rdf:predicate test-data:pred ;
+                                     rdf:object    test-data:obj ; ] ; ] ;
+----
+* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
+----
+    must:given [ a must:AnzoGraphmartDataset ;
+                 must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
+                 must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
+    ] .
+----
+=== Whens
+These are the actual SPARQL queries that you wish to test. Queries can be supplied as a string directly in the test or as a file containing the query. Only single When statements are currently supported.
+Mustrd does not derive the query type from the actual query, so it is necessary to provide this in the specification. Supported query types are SelectSparql, ConstructSparql and UpdateSparql.
+
+* *TextSparqlSource* - The SPARQL query is included in the test as a (multiline) string value for the property queryText.
+e.g.
+----
+    must:when  [ a must:TextSparqlSource ;
+                 must:queryText "SELECT ?s ?p ?o WHERE { ?s ?p ?o }" ;
+                 must:queryType must:SelectSparql ] ;
+----
+
+* *FileSparqlSource* - The SPARQL query is contained in a local file.
+e.g.
+----
+    must:when  [ a must:FileSparqlSource  ;
+                 must:file "test/data/construct.rq" ;
+                 must:queryType must:ConstructSparql  ; ] ;
+----
+* *FolderSparqlSource* - Similar to the file SPARQL source except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -w option on the command line.
+----
+    must:when  [ a must:FolderSparqlSource ;
+                 must:fileName "construct.rq" ;
+                 must:queryType must:ConstructSparql  ; ] ;
+----
+* *AnzoQueryBuilderDataset* - The query is saved in the Query Builder of an Anzo instance and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
+----
+   must:when  [ a must:AnzoQueryBuilderDataset ;
+                must:queryFolder "Mustrd";
+                must:queryName "mustrd-construct" ;
+                must:queryType must:ConstructSparql
+    ];
+----
+=== Thens
+Then clauses are used to specify the expected result dataset for the test. These datasets can be specified in the same way as <<Givens>> except that an extended set of dataset types is supported. For the tabular results of SELECT queries TabularDatasets are required and again can be in file format such as CSV, or an inline table within the specification.
+* *FileDataset* - The dataset is a local file containing serialised RDF or tabular data. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX, as well as tabular formats (.csv, .xls, .xlsx).
+----
+    must:then  [ a must:FileDataset ;
+                 must:file "test/data/thenSuccess.xlsx" ] .
+----
+----
+    must:then  [ a must:FileDataset ;
+                 must:file "test/data/thenSuccess.nt" ] .
+----
+* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -t option on the command line.
+----
+    must:then [ a must:FolderDataset ;
+                 must:fileName "then.ttl" ] ;
+----
+* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements e.g.
+----
+    must:then [ a must:StatementsDataset ;
+                 must:hasStatement [ a rdf:Statement ;
+                                     rdf:subject   test-data:sub ;
+                                     rdf:predicate test-data:pred ;
+                                     rdf:object    test-data:obj ; ] ; ] ;
+----
+* *TableDataset* - The contents of the table defined in RDF syntax within the specification.
+E.g. a table dataset consisting of a single row and three columns.
+----
+    must:then  [ a must:TableDataset ;
+                   must:hasRow [ must:hasBinding[
+                        must:variable "s" ;
+                        must:boundValue  test-data:sub ; ],
+                      [ must:variable "p" ;
+                        must:boundValue  test-data:pred ; ],
+                      [ must:variable "o" ;
+                        must:boundValue  test-data:obj ; ] ;
+               ] ; ] .
+----
+* *OrderedTableDataset* -  This is an extension of the TableDataset which allows the row order of the dataset to be specified using the SHACL order property to support the ORDER BY clause in SPARQL SELECT queries
+E.g. A table dataset consisting of two ordered rows and three columns.
+----
+    must:then  [ a must:OrderedTableDataset ;
+                 must:hasRow [ sh:order 1 ;
+                             must:hasBinding[ must:variable "s" ;
+                                        must:boundValue  test-data:sub1 ; ],
+                                      [ must:variable "p" ;
+                                        must:boundValue  test-data:pred1 ; ],
+                                      [ must:variable "o" ;
+                                        must:boundValue  test-data:obj1 ; ] ; ] ,
+                            [ sh:order 2 ;
+                             must:hasBinding[ must:variable "s" ;
+                                        must:boundValue  test-data:sub2 ; ],
+                                      [ must:variable "p" ;
+                                        must:boundValue  test-data:pred2 ; ],
+                                      [ must:variable "o" ;
+                                        must:boundValue  test-data:obj2 ; ] ; ] ;
+               ] .
+----
+* *EmptyTable* - This is used to indicate that we are expecting an empty result from a SPARQL SELECT query.
+----
+    must:then  [ a must:EmptyTable ] .
+----
+* *EmptyGraph* - Similar to EmptyTable but used to indicate that we are expecting an empty graph as a result from a SPARQL query.
+----
+    must:then  [ a must:EmptyGraph ] .
+----
+* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
+----
+    must:then [ a must:AnzoGraphmartDataset ;
+                must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
+                must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
+        ] .
+----
+== Configuring external triplestores
+The configuration file for external triplestores can be located outside of the project root as it is specified as an argument to the mustard module or as the -c option on the commandline when running run.py.
+
+It is anticipated that the external triplestore is running as mustrd is not configured to start them.
+
+Currently, the supported external triplestores are GraphDB and Anzo.
+
+The configuration file should be serialised RDF. An example in Turtle format is included below for GraphDB. For Anzo the *must:repository* value is replaced with a *must:gqeURI*.
+----
+@prefix must:      <https://mustrd.com/model/> .
+must:GraphDbConfig1  a must:GraphDbConfig ;
+        must:url "http://localhost";
+        must:port "7200";
+        must:inputGraph "http://localhost:7200/test-graph" ;
+        must:repository "mustrd" .
+----
+To avoid versioning secrets when you want to version triplestore configuration (for example in case you want to run mustrd in CI), you have to configure user/password in a different file.
+This file must be named as the triple store configuration file, but with "_secrets" just before the extension. For example triplestores.ttl -> triplestores_secrets.ttl
+Subjects in the two files must match, no need to redefine the type, for example:
+----
+@prefix must:      <https://mustrd.com/model/> .
+must:GraphDbConfig1  must:username 'test' ;
+              must:password 'test' .
+----
+
+== Additional Notes for Developers
+Mustrd remains very much under development. It is anticipated that additional functionality and triplestore support will be added over time. The project uses https://python-poetry.org/docs/[Poetry] to manage dependencies so it will be necessary to have this installed to contribute towards the project. The link contains instructions on how to install and use this.
+As the project is actually built from the requirements.txt file at the project root, it is necessary to export dependencies from poetry to this file before committing and pushing changes to the repository, using the following command.
+
+`poetry export -f requirements.txt --without-hashes > requirements.txt`
+
+
+
+// end::body[]