mustrd 0.2.1__py3-none-any.whl → 0.2.2__py3-none-any.whl

mustrd/README.adoc

@@ -22,183 +22,6 @@ For a brief explanation of the meaning of these options use the help option.
 
 Run `pytest` from the project root.
 
-== Creating your own Test Specifications
-
-If you have got this far then you are probably ready to create your own specifications to test your application SPARQL queries. These will be executed against the default RDFLib triplestore unless you configure one or more alternatives. The instructions for this are included in <<Configuring external triplestores>> below.
-
-=== Paths
-All paths are consired relative. That way mustrd tests can be versionned and shared easily.
-To get absolute path from relative path in a spec file, we prefix it with the first existing result in:
-1) Path where the spec is located
-2) spec_path defined in mustrd test configuration files or cmd line argument
-3) data_path defined in mustrd test configuration files or cmd line argument
-4) Mustrd folder: In case of default resources packaged with mustrd source (will be in venv when mustrd is called as library)
-We intentionally use the same method to build paths in all spec components to avoid confusion.
-
-=== Givens
-These are used to specify the dataset against which the SPARQL statement will be run.
-They can be generated from external sources such as an existing graph, or a file or folder containing serialised RDF. It is also possible to specify the dataset as reified RDF directly in the test step. Currently tabular data sources such as csv files or TableDatasets are not supported.
-Multiple given statements can be supplied and data is combined into a single dataset for the test.
-
-* *InheritedDataset* - This is where no data is specified but the existing data in the target graph is retained rather than being replaced with a defined set. This can be used to chain tests together or to perform checks on application data.
-----
-    must:given [ a must:InheritedDataset ] ;
-----
-* *FileDataset* - The dataset is a local file containing serialised RDF. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX. The data is used to replace any existing content in the target graph for the test.
-----
-    must:given [ a must:FileDataset ;
-                 must:file "test/data/given.ttl" . ] ;
-----
-* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -g option on the command line.
-----
-    must:given [ a must:FolderDataset ;
-                 must:fileName "given.ttl" ] ;
-----
-* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements. e.g.
-----
-    must:given [ a must:StatementsDataset ;
-                 must:hasStatement [ a rdf:Statement ;
-                                     rdf:subject   test-data:sub ;
-                                     rdf:predicate test-data:pred ;
-                                     rdf:object    test-data:obj ; ] ; ] ;
-----
-* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-    must:given [ a must:AnzoGraphmartDataset ;
-                 must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
-                 must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
-    ] .
-----
-=== Whens
-These are the actual SPARQL queries that you wish to test. Queries can be supplied as a string directly in the test or as a file containing the query. Only single When statements are currently supported.
-Mustrd does not derive the query type from the actual query, so it is necessary to provide this in the specification. Supported query types are SelectSparql, ConstructSparql and UpdateSparql.
-
-* *TextSparqlSource* - The SPARQL query is included in the test as a (multiline) string value for the property queryText.
-e.g.
-----
-    must:when  [ a must:TextSparqlSource ;
-                 must:queryText "SELECT ?s ?p ?o WHERE { ?s ?p ?o }" ;
-                 must:queryType must:SelectSparql ] ;
-----
-
-* *FileSparqlSource* - The SPARQL query is contained in a local file.
-e.g.
-----
-    must:when  [ a must:FileSparqlSource  ;
-                 must:file "test/data/construct.rq" ;
-                 must:queryType must:ConstructSparql  ; ] ;
-----
-* *FolderSparqlSource* - Similar to the file SPARQL source except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -w option on the command line.
-----
-    must:when  [ a must:FolderSparqlSource ;
-                 must:fileName "construct.rq" ;
-                 must:queryType must:ConstructSparql  ; ] ;
-----
-* *AnzoQueryBuilderDataset* - The query is saved in the Query Builder of an Anzo instance and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-   must:when  [ a must:AnzoQueryBuilderDataset ;
-                must:queryFolder "Mustrd";
-                must:queryName "mustrd-construct" ;
-                must:queryType must:ConstructSparql
-    ];
-----
-=== Thens
-Then clauses are used to specify the expected result dataset for the test. These datasets can be specified in the same way as <<Givens>> except that an extended set of dataset types is supported. For the tabular results of SELECT queries TabularDatasets are required and again can be in file format such as CSV, or an inline table within the specification.
-* *FileDataset* - The dataset is a local file containing serialised RDF or tabular data. The formats supported are the same as those for the RDFLib Graph().parse function i.e. Turtle (.ttl), NTriples (.nt), N3 (.n3), RDF/XML (.xml) and TriX, as well as tabular formats (.csv, .xls, .xlsx).
-----
-    must:then  [ a must:FileDataset ;
-                 must:file "test/data/thenSuccess.xlsx" ] .
-----
-----
-    must:then  [ a must:FileDataset ;
-                 must:file "test/data/thenSuccess.nt" ] .
-----
-* *FolderDataset* - Very similar to the file dataset except that the location of the file is passed to the test specification as an argument from the caller. i.e. the -t option on the command line.
-----
-    must:then [ a must:FolderDataset ;
-                 must:fileName "then.ttl" ] ;
-----
-* *StatementsDataset* - The dataset is defined within the test in the form of reified RDF statements e.g.
-----
-    must:then [ a must:StatementsDataset ;
-                 must:hasStatement [ a rdf:Statement ;
-                                     rdf:subject   test-data:sub ;
-                                     rdf:predicate test-data:pred ;
-                                     rdf:object    test-data:obj ; ] ; ] ;
-----
-* *TableDataset* - The contents of the table defined in RDF syntax within the specification.
-E.g. a table dataset consisting of a single row and three columns.
-----
-    must:then  [ a must:TableDataset ;
-                   must:hasRow [ must:hasBinding[
-                        must:variable "s" ;
-                        must:boundValue  test-data:sub ; ],
-                      [ must:variable "p" ;
-                        must:boundValue  test-data:pred ; ],
-                      [ must:variable "o" ;
-                        must:boundValue  test-data:obj ; ] ;
-               ] ; ] .
-----
-* *OrderedTableDataset* -  This is an extension of the TableDataset which allows the row order of the dataset to be specified using the SHACL order property to support the ORDER BY clause in SPARQL SELECT queries
-E.g. A table dataset consisting of two ordered rows and three columns.
-----
-    must:then  [ a must:OrderedTableDataset ;
-                 must:hasRow [ sh:order 1 ;
-                             must:hasBinding[ must:variable "s" ;
-                                        must:boundValue  test-data:sub1 ; ],
-                                      [ must:variable "p" ;
-                                        must:boundValue  test-data:pred1 ; ],
-                                      [ must:variable "o" ;
-                                        must:boundValue  test-data:obj1 ; ] ; ] ,
-                            [ sh:order 2 ;
-                             must:hasBinding[ must:variable "s" ;
-                                        must:boundValue  test-data:sub2 ; ],
-                                      [ must:variable "p" ;
-                                        must:boundValue  test-data:pred2 ; ],
-                                      [ must:variable "o" ;
-                                        must:boundValue  test-data:obj2 ; ] ; ] ;
-               ] .
-----
-* *EmptyTable* - This is used to indicate that we are expecting an empty result from a SPARQL SELECT query.
-----
-    must:then  [ a must:EmptyTable ] .
-----
-* *EmptyGraph* - Similar to EmptyTable but used to indicate that we are expecting an empty graph as a result from a SPARQL query.
-----
-    must:then  [ a must:EmptyGraph ] .
-----
-* *AnzoGraphmartDataset* - The dataset is contained in an Anzo graphmart and needs to be retrieved from there. The Anzo instance containing the dataset needs to be indicated in the configuration file as documented in <<Configuring external triplestores>>.
-----
-    must:then [ a must:AnzoGraphmartDataset ;
-                must:graphmart "http://cambridgesemantics.com/Graphmart/43445aeadf674e09818c81cf7049e46a";
-                must:layer "http://cambridgesemantics.com/Layer/33b97531d7e148748b75e4e3c6bbf164";
-        ] .
-----
-== Configuring external triplestores
-The configuration file for external triplestores can be located outside of the project root as it is specified as an argument to the mustard module or as the -c option on the commandline when running run.py.
-
-It is anticipated that the external triplestore is running as mustrd is not configured to start them.
-
-Currently, the supported external triplestores are GraphDB and Anzo.
-
-The configuration file should be serialised RDF. An example in Turtle format is included below for GraphDB. For Anzo the *must:repository* value is replaced with a *must:gqeURI*.
-----
-@prefix must:      <https://mustrd.com/model/> .
-must:GraphDbConfig1  a must:GraphDbConfig ;
-        must:url "http://localhost";
-        must:port "7200";
-        must:inputGraph "http://localhost:7200/test-graph" ;
-        must:repository "mustrd" .
-----
-To avoid versioning secrets when you want to version triplestore configuration (for example in case you want to run mustrd in CI), you have to configure user/password in a different file.
-This file must be named as the triple store configuration file, but with "_secrets" just before the extension. For example triplestores.ttl -> triplestores_secrets.ttl
-Subjects in the two files must match, no need to redefine the type, for example:
-----
-@prefix must:      <https://mustrd.com/model/> .
-must:GraphDbConfig1  must:username 'test' ;
-              must:password 'test' .
-----
-
 == Additional Notes for Developers
 Mustrd remains very much under development. It is anticipated that additional functionality and triplestore support will be added over time. The project uses https://python-poetry.org/docs/[Poetry] to manage dependencies so it will be necessary to have this installed to contribute towards the project. The link contains instructions on how to install and use this.
 As the project is actually built from the requirements.txt file at the project root, it is necessary to export dependencies from poetry to this file before committing and pushing changes to the repository, using the following command.

mustrd/TestResult.py

@@ -43,7 +43,7 @@ class testStatus(Enum):
     SKIPPED = "skipped"
 
 
-TEMPLATE_FOLDER =  Path(os.path.join(get_mustrd_root(), "templates/"))
+TEMPLATE_FOLDER = Path(os.path.join(get_mustrd_root(), "templates/"))
 
 
 RESULT_LIST_MD_TEMPLATE = "md_ResultList_template.jinja"

mustrd/mustrd.py

@@ -46,12 +46,12 @@ import json
 from pandas import DataFrame
 
 from .spec_component import TableThenSpec, parse_spec_component, WhenSpec, ThenSpec
-from .utils import  is_json,get_mustrd_root
+from .utils import is_json, get_mustrd_root
 from colorama import Fore, Style
 from tabulate import tabulate
 from collections import defaultdict
 from pyshacl import validate
-import logging 
+import logging
 from http.client import HTTPConnection
 from .steprunner import upload_given, run_when
 
@@ -73,6 +73,7 @@ def debug_requests_on():
     requests_log.setLevel(logging.DEBUG)
     requests_log.propagate = True
 
+
 def debug_requests_off():
     '''Switches off logging of the requests module, might be some side-effects'''
     HTTPConnection.debuglevel = 0
@@ -84,8 +85,10 @@ def debug_requests_off():
     requests_log.setLevel(logging.WARNING)
     requests_log.propagate = False
 
+
 debug_requests_off()
 
+
 @dataclass
 class Specification:
     spec_uri: URIRef
@@ -234,25 +237,18 @@ def validate_specs(run_config: dict, triple_stores: List, shacl_graph: Graph, on
             if len(error_messages) > 0:
                 error_messages.sort()
                 error_message = "\n".join(msg for msg in error_messages)
-                invalid_specs += [SpecSkipped(subject_uri, triple_store["type"], error_message, file.name) for triple_store in
-                                triple_stores]
+                invalid_specs += [SpecSkipped(subject_uri, triple_store["type"], error_message, file.name)
+                                  for triple_store in triple_stores]
             else:
                 subject_uris.add(subject_uri)
                 this_spec_graph = Graph()
                 this_spec_graph.parse(file)
                 spec_uris_in_this_file = list(this_spec_graph.subjects(RDF.type, MUST.TestSpec))
                 for spec in spec_uris_in_this_file:
-                    # print(f"adding {tripleToAdd}")
                     this_spec_graph.add([spec, MUST.specSourceFile, Literal(file)])
                     this_spec_graph.add([spec, MUST.specFileName, Literal(file.name)])
-                # print(f"beforeadd: {spec_graph}" )
-                # print(f"beforeadd: {str(this_spec_graph.serialize())}" )
                 spec_graph += this_spec_graph
 
-
-    sourceFiles = list(spec_graph.subject_objects(MUST.specSourceFile))
-    # print(f"sourceFiles: {sourceFiles}")
-
     valid_spec_uris = list(spec_graph.subjects(RDF.type, MUST.TestSpec))
 
     if focus_uris:
@@ -264,7 +260,7 @@ def validate_specs(run_config: dict, triple_stores: List, shacl_graph: Graph, on
         log.info(f"Collected {len(focus_uris)} focus test spec(s)")
         return focus_uris, spec_graph, invalid_focus_specs
     else:
-        log.info(f"Collected {len(valid_spec_uris)} valid test spec(s)")   
+        log.info(f"Collected {len(valid_spec_uris)} valid test spec(s)")
         return valid_spec_uris, spec_graph, invalid_specs
 
 
@@ -276,14 +272,16 @@ def get_specs(spec_uris: List[URIRef], spec_graph: Graph, triple_stores: List[di
         for triple_store in triple_stores:
             if "error" in triple_store:
                 log.error(f"{triple_store['error']}. No specs run for this triple store.")
-                skipped_results += [SpecSkipped(spec_uri, triple_store['type'], triple_store['error'], get_spec_file(spec_uri, spec_graph)) for spec_uri in
+                skipped_results += [SpecSkipped(spec_uri, triple_store['type'], triple_store['error'],
+                                                get_spec_file(spec_uri, spec_graph)) for spec_uri in
                                     spec_uris]
             else:
                 for spec_uri in spec_uris:
                     try:
                         specs += [get_spec(spec_uri, spec_graph, run_config, triple_store)]
                     except (ValueError, FileNotFoundError, ConnectionError) as e:
-                        skipped_results += [SpecSkipped(spec_uri, triple_store['type'], e, get_spec_file(spec_uri, spec_graph))]
+                        skipped_results += [SpecSkipped(spec_uri, triple_store['type'],
+                                                        e, get_spec_file(spec_uri, spec_graph))]
 
     except (BadSyntax, FileNotFoundError) as e:
         template = "An exception of type {0} occurred when trying to parse the triple store configuration file. " \
@@ -303,25 +301,28 @@ def run_specs(specs) -> List[SpecResult]:
         results.append(run_spec(specification))
     return results
 
+
 def get_spec_file(spec_uri: URIRef, spec_graph: Graph):
-    return str(spec_graph.value(subject = spec_uri, predicate = MUST.specFileName, default = "default.mustrd.ttl"))
+    return str(spec_graph.value(subject=spec_uri, predicate=MUST.specFileName, default="default.mustrd.ttl"))
+
 
 def get_spec(spec_uri: URIRef, spec_graph: Graph, run_config: dict, mustrd_triple_store: dict = None) -> Specification:
     try:
-        if mustrd_triple_store is None:
+        if not mustrd_triple_store:
             mustrd_triple_store = {"type": TRIPLESTORE.RdfLib}
         components = []
         for predicate in MUST.given, MUST.when, MUST.then:
             components.append(parse_spec_component(subject=spec_uri,
-                                                predicate=predicate,
-                                                spec_graph=spec_graph,
-                                                run_config=run_config,
-                                                mustrd_triple_store=mustrd_triple_store))
+                                                   predicate=predicate,
+                                                   spec_graph=spec_graph,
+                                                   run_config=run_config,
+                                                   mustrd_triple_store=mustrd_triple_store))
 
         spec_file_name = get_spec_file(spec_uri, spec_graph)
         # https://github.com/Semantic-partners/mustrd/issues/92
-        return Specification(spec_uri, mustrd_triple_store, components[0].value, components[1], components[2], spec_file_name)
-    
+        return Specification(spec_uri, mustrd_triple_store,
+                             components[0].value, components[1], components[2], spec_file_name)
+
     except (ValueError, FileNotFoundError) as e:
         template = "An exception of type {0} occurred. Arguments:\n{1!r}"
         message = template.format(type(e).__name__, e.args)
@@ -333,7 +334,7 @@ def get_spec(spec_uri: URIRef, spec_graph: Graph, run_config: dict, mustrd_tripl
 
 
 def check_result(spec, result):
-    if type(spec.then) == TableThenSpec:
+    if isinstance(spec.then, TableThenSpec):
         return table_comparison(result, spec)
     else:
         graph_compare = graph_comparison(spec.then.value, result)
@@ -383,27 +384,30 @@ def run_spec(spec: Specification) -> SpecResult:
     #     if type(mustrd_triple_store) == MustrdAnzo and close_connection:
     #         mustrd_triple_store.clear_graph()
 
+
 def get_triple_store_graph(triple_store_graph_path: Path, secrets: str):
     if secrets:
-        return Graph().parse(triple_store_graph_path).parse(data = secrets)
+        return Graph().parse(triple_store_graph_path).parse(data=secrets)
     else:
-        secret_path = triple_store_graph_path.parent / Path(triple_store_graph_path.stem + "_secrets" + triple_store_graph_path.suffix)
+        secret_path = triple_store_graph_path.parent / Path(triple_store_graph_path.stem +
+                                                            "_secrets" + triple_store_graph_path.suffix)
         return Graph().parse(triple_store_graph_path).parse(secret_path)
-    
+
 
 def get_triple_stores(triple_store_graph: Graph) -> list[dict]:
     triple_stores = []
     shacl_graph = Graph().parse(Path(os.path.join(get_mustrd_root(), "model/triplestoreshapes.ttl")))
     ont_graph = Graph().parse(Path(os.path.join(get_mustrd_root(), "model/triplestoreOntology.ttl")))
     conforms, results_graph, results_text = validate(
-            data_graph= triple_store_graph,
-            shacl_graph = shacl_graph,
-            ont_graph  = ont_graph,
-            advanced= True,
-            inference= 'none'
+            data_graph=triple_store_graph,
+            shacl_graph=shacl_graph,
+            ont_graph=ont_graph,
+            advanced=True,
+            inference='none'
         )
     if not conforms:
-        raise ValueError(f"Triple store configuration not conform to the shapes. SHACL report: {results_text}", results_graph)
+        raise ValueError(f"Triple store configuration not conform to the shapes. SHACL report: {results_text}",
+                         results_graph)
     for triple_store_config, rdf_type, triple_store_type in triple_store_graph.triples((None, RDF.type, None)):
         triple_store = {}
         triple_store["type"] = triple_store_type
@@ -413,15 +417,18 @@ def get_triple_stores(triple_store_graph: Graph) -> list[dict]:
             triple_store["url"] = triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.url)
             triple_store["port"] = triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.port)
             try:
-                triple_store["username"] = str(triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.username))
-                triple_store["password"] = str(triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.password))
+                triple_store["username"] = str(triple_store_graph.value(subject=triple_store_config,
+                                                                        predicate=TRIPLESTORE.username))
+                triple_store["password"] = str(triple_store_graph.value(subject=triple_store_config,
+                                                                        predicate=TRIPLESTORE.password))
             except (FileNotFoundError, ValueError) as e:
                 triple_store["error"] = e
-            triple_store["gqe_uri"] = triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.gqeURI)
+            triple_store["gqe_uri"] = triple_store_graph.value(subject=triple_store_config,
+                                                               predicate=TRIPLESTORE.gqeURI)
             triple_store["input_graph"] = triple_store_graph.value(subject=triple_store_config,
                                                                    predicate=TRIPLESTORE.inputGraph)
             triple_store["output_graph"] = triple_store_graph.value(subject=triple_store_config,
-                                                                   predicate=TRIPLESTORE.outputGraph)
+                                                                    predicate=TRIPLESTORE.outputGraph)
             try:
                 check_triple_store_params(triple_store, ["url", "port", "username", "password", "input_graph"])
             except ValueError as e:
@@ -431,8 +438,10 @@ def get_triple_stores(triple_store_graph: Graph) -> list[dict]:
             triple_store["url"] = triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.url)
             triple_store["port"] = triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.port)
             try:
-                triple_store["username"] = str(triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.username))
-                triple_store["password"] = str(triple_store_graph.value(subject=triple_store_config, predicate=TRIPLESTORE.password))
+                triple_store["username"] = str(triple_store_graph.value(subject=triple_store_config,
+                                                                        predicate=TRIPLESTORE.username))
+                triple_store["password"] = str(triple_store_graph.value(subject=triple_store_config,
+                                                                        predicate=TRIPLESTORE.password))
             except (FileNotFoundError, ValueError) as e:
                 log.error(f"Credential retrieval failed {e}")
                 triple_store["error"] = e
@@ -461,11 +470,9 @@ def check_triple_store_params(triple_store: dict, required_params: List[str]):
 
 def get_credential_from_file(triple_store_name: URIRef, credential: str, config_path: Literal) -> str:
     log.info(f"get_credential_from_file {triple_store_name}, {credential}, {config_path}")
-    if config_path is None:
+    if not config_path:
         raise ValueError(f"Cannot establish connection defined in {triple_store_name}. "
                          f"Missing required parameter: {credential}.")
-    # if os.path.isrelative(config_path)
-    # project_root = get_project_root()
     path = Path(config_path)
     log.info(f"get_credential_from_file {path}")
 
@@ -480,6 +487,7 @@ def get_credential_from_file(triple_store_name: URIRef, credential: str, config_
         raise ValueError(f"Error reading credentials config file: {e}")
     return config[str(triple_store_name)][credential]
 
+
 # Convert sparql json query results as defined in https://www.w3.org/TR/rdf-sparql-json-res/
 def json_results_to_panda_dataframe(result: str) -> pandas.DataFrame:
     json_result = json.loads(result)
@@ -534,7 +542,8 @@ def table_comparison(result: str, spec: Specification) -> SpecResult:
 
             # Scenario 1: expected no result but got a result
             if then.empty:
-                message = f"Expected 0 row(s) and 0 column(s), got {df.shape[0]} row(s) and {round(df.shape[1] / 2)} column(s)"
+                message = f"""Expected 0 row(s) and 0 column(s),
+                got {df.shape[0]} row(s) and {round(df.shape[1] / 2)} column(s)"""
                 empty_then = create_empty_dataframe_with_columns(df)
                 df_diff = empty_then.compare(df, result_names=("expected", "actual"))
 
@@ -546,14 +555,6 @@ def table_comparison(result: str, spec: Specification) -> SpecResult:
                 if ordered_result is True and not spec.then.ordered:
                     message += ". Actual result is ordered, must:then must contain sh:order on every row."
                     return SelectSpecFailure(spec.spec_uri, spec.triple_store["type"], None, message)
-                    # if df.shape == then.shape and (df.columns == then.columns).all():
-                    #     df_diff = then.compare(df, result_names=("expected", "actual"))
-                    #     if df_diff.empty:
-                    #         df_diff = df
-                    #         print(df_diff.to_markdown())
-                    # else:
-                    #     df_diff = construct_df_diff(df, then)
-                    #     print(df_diff.to_markdown())
                 else:
                     if len(columns) == len(then.columns):
                         if sorted_columns == sorted_then_cols:
@@ -579,15 +580,15 @@ def table_comparison(result: str, spec: Specification) -> SpecResult:
 
             if then.empty:
                 # Scenario 3: expected no result, got no result
-                message = f"Expected 0 row(s) and 0 column(s), got 0 row(s) and 0 column(s)"
+                message = "Expected 0 row(s) and 0 column(s), got 0 row(s) and 0 column(s)"
                 df = pandas.DataFrame()
             else:
                 # Scenario 4: expected a result, but got an empty result
-                message = f"Expected {then.shape[0]} row(s) and {round(then.shape[1] / 2)} column(s), got 0 row(s) and 0 column(s)"
+                message = f"""Expected {then.shape[0]} row(s)
+                              and {round(then.shape[1] / 2)} column(s), got 0 row(s) and 0 column(s)"""
                 then = then[sorted_then_cols]
                 df = create_empty_dataframe_with_columns(then)
             df_diff = then.compare(df, result_names=("expected", "actual"))
-            print(df_diff.to_markdown())
 
         if df_diff.empty:
             if warning:
@@ -595,13 +596,8 @@ def table_comparison(result: str, spec: Specification) -> SpecResult:
             else:
                 return SpecPassed(spec.spec_uri, spec.triple_store["type"])
         else:
-            # message += f"\nexpected:\n{then}\nactual:{df}"
+            log.error("\n" + df_diff.to_markdown())
             log.error(message)
-            # print(spec.spec_uri)
-            # print("actual:")
-            # print(then)
-            # print("expected:")
-            # print(df)
             return SelectSpecFailure(spec.spec_uri, spec.triple_store["type"], df_diff, message)
 
     except ParseException as e:
@@ -622,18 +618,18 @@ def graph_comparison(expected_graph: Graph, actual_graph: Graph) -> GraphCompari
 
 def get_then_update(spec_uri: URIRef, spec_graph: Graph) -> Graph:
     then_query = f"""
-    prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
+    prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 
     CONSTRUCT {{ ?s ?p ?o }}
     {{
-        <{spec_uri}> <{MUST.then}> 
+        <{spec_uri}> <{MUST.then}>
             a <{MUST.StatementsDataset}> ;
             <{MUST.hasStatement}> [
                 a rdf:Statement ;
                 rdf:subject ?s ;
                 rdf:predicate ?p ;
                 rdf:object ?o ;
-            ] ; ] 
+            ] ; ]
     }}
     """
     expected_results = spec_graph.query(then_query).graph
@@ -707,7 +703,7 @@ def create_empty_dataframe_with_columns(df: pandas.DataFrame) -> pandas.DataFram
 
 
 def review_results(results: List[SpecResult], verbose: bool) -> None:
-    print("===== Result Overview =====")
+    log.info("===== Result Overview =====")
     # Init dictionaries
     status_dict = defaultdict(lambda: defaultdict(int))
     status_counts = defaultdict(lambda: defaultdict(int))
@@ -723,7 +719,8 @@ def review_results(results: List[SpecResult], verbose: bool) -> None:
 
     # Convert dictionaries to list for tabulate
     table_rows = [[spec_uri] + [
-        f"{colours.get(status_dict[spec_uri][triple_store], Fore.RED)}{status_dict[spec_uri][triple_store].__name__}{Style.RESET_ALL}"
+        f"""{colours.get(status_dict[spec_uri][triple_store], Fore.RED)}
+        {status_dict[spec_uri][triple_store].__name__}{Style.RESET_ALL}"""
         for triple_store in triple_stores] for spec_uri in set(status_dict.keys())]
 
     status_rows = [[f"{colours.get(status, Fore.RED)}{status.__name__}{Style.RESET_ALL}"] +
@@ -731,8 +728,8 @@ def review_results(results: List[SpecResult], verbose: bool) -> None:
                     for triple_store in triple_stores] for status in set(statuses)]
 
     # Display tables with tabulate
-    print(tabulate(table_rows, headers=['Spec Uris / triple stores'] + triple_stores, tablefmt="pretty"))
-    print(tabulate(status_rows, headers=['Status / triple stores'] + triple_stores, tablefmt="pretty"))
+    log.info(tabulate(table_rows, headers=['Spec Uris / triple stores'] + triple_stores, tablefmt="pretty"))
+    log.info(tabulate(status_rows, headers=['Status / triple stores'] + triple_stores, tablefmt="pretty"))
 
     pass_count = statuses.count(SpecPassed)
     warning_count = statuses.count(SpecPassedWithWarning)
@@ -748,40 +745,34 @@ def review_results(results: List[SpecResult], verbose: bool) -> None:
         overview_colour = Fore.GREEN
 
     logger_setup.flush()
-    print(f"{overview_colour}===== {fail_count} failures, {skipped_count} skipped, {Fore.GREEN}{pass_count} passed, "
+    log.info(f"{overview_colour}===== {fail_count} failures, {skipped_count} skipped, {Fore.GREEN}{pass_count} passed, "
           f"{overview_colour}{warning_count} passed with warnings =====")
 
     if verbose and (fail_count or warning_count or skipped_count):
         for res in results:
-            if type(res) == UpdateSpecFailure:
-                print(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
-                print(f"{Fore.BLUE} In Expected Not In Actual:")
-                print(res.graph_comparison.in_expected_not_in_actual.serialize(format="ttl"))
-                print()
-                print(f"{Fore.RED} in_actual_not_in_expected")
-                print(res.graph_comparison.in_actual_not_in_expected.serialize(format="ttl"))
-                print(f"{Fore.GREEN} in_both")
-                print(res.graph_comparison.in_both.serialize(format="ttl"))
-
-            if type(res) == SelectSpecFailure:
-                print(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
-                print(res.message)
-                print(res.table_comparison.to_markdown())
-            if type(res) == ConstructSpecFailure or type(res) == UpdateSpecFailure:
-                print(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
-            if type(res) == SpecPassedWithWarning:
-                print(f"{Fore.YELLOW}Passed with warning {res.spec_uri} {res.triple_store}")
-                print(res.warning)
-            if type(res) == TripleStoreConnectionError or type(res) == SparqlExecutionError or \
-                    type(res) == SparqlParseFailure:
-                print(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
-                print(res.exception)
-            if type(res) == SpecSkipped:
-                print(f"{Fore.YELLOW}Skipped {res.spec_uri} {res.triple_store}")
-                print(res.message)
-
-
-
-    
-
-
+            if isinstance(res, UpdateSpecFailure):
+                log.info(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
+                log.info(f"{Fore.BLUE} In Expected Not In Actual:")
+                log.info(res.graph_comparison.in_expected_not_in_actual.serialize(format="ttl"))
+                log.info()
+                log.info(f"{Fore.RED} in_actual_not_in_expected")
+                log.info(res.graph_comparison.in_actual_not_in_expected.serialize(format="ttl"))
+                log.info(f"{Fore.GREEN} in_both")
+                log.info(res.graph_comparison.in_both.serialize(format="ttl"))
+
+            if isinstance(res, SelectSpecFailure):
+                log.info(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
+                log.info(res.message)
+                log.info(res.table_comparison.to_markdown())
+            if isinstance(res, ConstructSpecFailure) or isinstance(res, UpdateSpecFailure):
+                log.info(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
+            if isinstance(res, SpecPassedWithWarning):
+                log.info(f"{Fore.YELLOW}Passed with warning {res.spec_uri} {res.triple_store}")
+                log.info(res.warning)
+            if isinstance(res, TripleStoreConnectionError) or type(res, SparqlExecutionError) or \
+                    isinstance(res, SparqlParseFailure):
+                log.info(f"{Fore.RED}Failed {res.spec_uri} {res.triple_store}")
+                log.info(res.exception)
+            if isinstance(res, SpecSkipped):
+                log.info(f"{Fore.YELLOW}Skipped {res.spec_uri} {res.triple_store}")
+                log.info(res.message)