eva-sub-cli 0.4.12__tar.gz → 0.4.14__tar.gz

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eva_sub_cli
-Version: 0.4.12
+Version: 0.4.14
 Summary: EBI EVA - validation and submission command line tool
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console

eva_sub_cli-0.4.14/docs/installation.md

@@ -0,0 +1,70 @@
+# Installation
+
+There are two main ways to install eva-sub-cli, either through [conda](#1-conda) or [pip](#2-pip).
+Conda is the most straightforward and recommended method, as it will install all dependencies for you.
+
+Otherwise you can install using pip, in which case you will need to handle non-Python dependencies separately.
+You can do this either using Docker or natively (i.e. installing by yourself).
+Note that pip using Docker is currently the only installation method available for Windows users.
+
+We encourage users to install the most recent version, as we are constantly improving the tool. You can check that you
+have the latest version installed by running `eva-sub-cli.py --version` and comparing against the
+[release page](https://github.com/EBIvariation/eva-sub-cli/releases).
+
+All installation methods require Python 3.8 or above.
+
+## 1. Conda
+
+View our [video tutorial on conda installation](https://embl-ebi.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=aa82710e-4401-4074-b6b0-b346016bb14e).
+
+The most straightforward way to install eva-sub-cli and its dependencies is through conda.
+For instance, the following commands install eva-sub-cli in a new environment called `eva`, activate the environment, and print
+the help message:
+```bash
+conda create -n eva -c conda-forge -c bioconda eva-sub-cli
+conda activate eva
+eva-sub-cli.py --help
+````
+
+To upgrade to the newest version, run `conda update eva-sub-cli`.
+
+## 2. Pip
+
+View our [video tutorial on pip installation](https://embl-ebi.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=2e6828ed-31c2-4621-8eb5-b34701484aae).
+
+You can also install eva-sub-cli using pip. 
+The following commands will create a new virtual environment called `eva`, activate the environment, and install eva-sub-cli:
+```bash
+python3 -m venv eva
+source ./eva/bin/activate
+pip install eva-sub-cli
+```
+
+You can verify the installation by printing the help message:
+```bash
+eva-sub-cli.py --help
+```
+
+For external (non-Python) dependencies, you will need to install them either [via Docker](#docker) or [natively](#native).
+These methods are described below.
+
+To upgrade to the newest version of the tool, run `pip install --upgrade eva-sub-cli`.
+
+### Docker
+
+Docker provides an easy way to run eva-sub-cli without installing dependencies separately.
+For this method, all you need is for [Docker](https://docs.docker.com/engine/install/) to be installed and running in the background.
+
+When you run eva-sub-cli, you will need to [specify execution using Docker](how_to_submit.md#running-using-docker).
+The tool will then automatically run using a Docker container with all dependencies pre-installed.
+
+### Native
+
+For those requiring more flexibility or working on systems where Docker is not available, you can also install the 
+external dependencies on your own. These are:
+* [Nextflow](https://www.nextflow.io/docs/latest/getstarted.html) 21.10+
+* [biovalidator](https://github.com/elixir-europe/biovalidator) 2.1.0+
+* [vcf-validator](https://github.com/EBIvariation/vcf-validator) 0.9.7+
+
+Install each of these and ensure they are included in your PATH.
+This will allow eva-sub-cli to be executed natively, i.e. directly on your machine.

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/_version.py

@@ -1,7 +1,14 @@
 # file generated by setuptools-scm
 # don't change, don't track in version control
 
-__all__ = ["__version__", "__version_tuple__", "version", "version_tuple"]
+__all__ = [
+    "__version__",
+    "__version_tuple__",
+    "version",
+    "version_tuple",
+    "__commit_id__",
+    "commit_id",
+]
 
 TYPE_CHECKING = False
 if TYPE_CHECKING:
@@ -9,13 +16,19 @@ if TYPE_CHECKING:
     from typing import Union
 
     VERSION_TUPLE = Tuple[Union[int, str], ...]
+    COMMIT_ID = Union[str, None]
 else:
     VERSION_TUPLE = object
+    COMMIT_ID = object
 
 version: str
 __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
+commit_id: COMMIT_ID
+__commit_id__: COMMIT_ID
 
-__version__ = version = '0.4.12'
-__version_tuple__ = version_tuple = (0, 4, 12)
+__version__ = version = '0.4.14'
+__version_tuple__ = version_tuple = (0, 4, 14)
+
+__commit_id__ = commit_id = 'g1808b05b1'

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/etc/eva_schema.json

@@ -324,18 +324,7 @@
             "description": "BioSample accession of pre-existing sample. Only use for preregistered samples"
           },
           "bioSampleObject": {
-            "type": "object",
-            "anyOf": [
-              {
-                "$ref": "eva-biosamples.json"
-              },
-              {
-                "$ref": "https://www.ebi.ac.uk/biosamples/schema-store/registry/schemas/ERC000011"
-              },
-              {
-                "$ref": "https://www.ebi.ac.uk/biosamples/schemas/certification/plant-miappe.json"
-              }
-            ]
+            "$ref": "https://www.ebi.ac.uk/biosamples/schema-store/registry/schemas/ERC000011"
           }
         }
       }

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/etc/spreadsheet2json_conf.yaml

@@ -81,6 +81,8 @@ Sample:
   required:
     Analysis Alias: analysisAlias
     Sample Name in VCF: sampleInVCF
+  # Sample property names taken from ERC000011 where present
+  # See https://www.ebi.ac.uk/biosamples/schema-store/registry/schemas/ERC000011
   optional:
     Sample Accession: bioSampleAccession
     BioSample Name: bioSampleName
@@ -91,37 +93,37 @@ Sample:
     Subject: subject
     Derived From: derivedFrom
     Taxonomy ID: taxId
-    Scientific Name: scientificName
+    Scientific Name: scientific name
     Common Name: commonName
-    mating_type: matingType
+    mating_type: mating_type
     sex: sex
     population: population
-    cell_type: cellType
-    dev_stage: devStage
+    cell_type: cell_type
+    dev_stage: dev_stage
     germline: germline
-    tissue_lib: tissueLib
-    tissue_type: tissueType
-    bio_material: BioMaterial
-    culture_collection: cultureCollection
-    specimen_voucher: specimenVoucher
-    collected_by: collectedBy
-    collection_date: collectionDate
-    geographic location (country and/or sea): geographicLocationCountrySea
-    geographic location (region and locality): geographicLocationRegion
+    tissue_lib: tissue_lib
+    tissue_type: tissue_type
+    bio_material: bio_material
+    culture_collection: culture_collection
+    specimen_voucher: specimen_voucher
+    collected_by: collected_by
+    collection_date: collection date
+    geographic location (country and/or sea): geographic location (country and/or sea)
+    geographic location (region and locality): geographic location (region and locality)
     host: host
-    identified_by: identifiedBy
-    isolation_source: isolationSource
-    lat_lon: latLon
-    lab_host: LabHost
-    environmental_sample: environmentalSample
+    identified_by: identified_by
+    isolation_source: isolation_source
+    lat_lon: lat_lon
+    lab_host: lab_host
+    environmental_sample: environmental_sample
     cultivar: cultivar
     ecotype: ecotype
     isolate: isolate
     strain: strain
-    sub_species: subSpecies
+    sub_species: sub_species
     variety: variety
-    sub_strain: subStrain
-    cell_line: cellType
+    sub_strain: sub_strain
+    cell_line: cell_line
     serotype: serotype
     serovar: serovar
   cast:

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/executables/cli.py

@@ -22,24 +22,47 @@ from eva_sub_cli.file_utils import is_submission_dir_writable, DirLockError, Dir
 
 
 def validate_command_line_arguments(args, argparser):
+    fail = False
     if (args.vcf_files and not args.reference_fasta) or (not args.vcf_files and args.reference_fasta):
         print("When using --vcf_files and --reference_fasta, both need to be specified")
-        argparser.print_usage()
-        sys.exit(1)
+        fail = True
+
+    if args.vcf_files:
+        for vcf_file in args.vcf_files:
+            if not os.path.isfile(vcf_file):
+                print(f"VCF file {vcf_file} is not a file")
+                fail = True
+
+    if args.reference_fasta:
+        if not os.path.isfile(args.reference_fasta):
+            print(f"Fasta file {args.reference_fasta} is not a file")
+            fail = True
+
+    if args.metadata_xlsx:
+        if not os.path.isfile(args.metadata_xlsx):
+            print(f"Spreadsheet file {args.metadata_xlsx} is not a file")
+            fail = True
+
+    if args.metadata_json:
+        if not os.path.isfile(args.metadata_json):
+            print(f"JSON file {args.metadata_json} is not a file")
+            fail = True
 
     if SUBMIT in args.tasks and (
             not (args.username or os.environ.get('ENAWEBINACCOUNT')) or
             not (args.password or os.environ.get('ENAWEBINPASSWORD'))):
         print("To submit your data, you need to provide a Webin username and password")
-        argparser.print_usage()
-        sys.exit(1)
+        fail = True
 
     if not is_submission_dir_writable(args.submission_dir):
         print(f"'{args.submission_dir}' does not have write permissions or is not a directory.")
-        sys.exit(1)
+        fail = True
 
     if args.nextflow_config and not os.path.isfile(args.nextflow_config):
         print(f"'{args.nextflow_config}' is not a file or does not exist.")
+        fail = True
+
+    if fail:
         argparser.print_usage()
         sys.exit(1)
 

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/executables/xlsx2json.py

@@ -9,6 +9,8 @@ from ebi_eva_common_pyutils.logger import logging_config
 from openpyxl.reader.excel import load_workbook
 from openpyxl.workbook import Workbook
 
+from eva_sub_cli.orchestrator import get_sub_cli_version, get_sub_cli_github_tags
+
 WORKSHEETS_KEY_NAME = 'worksheets'
 REQUIRED_HEADERS_KEY_NAME = 'required'
 OPTIONAL_HEADERS_KEY_NAME = 'optional'
@@ -249,7 +251,8 @@ class XlsxParser:
         # If the file could not be loaded at all, return without generating JSON.
         if not self.file_loaded:
             return
-        json_data = {}
+        # Include which version of the EVA schema we are using
+        json_data = {'$schema': self.get_json_schema_link()}
         for title in self.xlsx_conf[WORKSHEETS_KEY_NAME]:
             self.active_worksheet = title
             if title == PROJECT:
@@ -270,6 +273,14 @@ class XlsxParser:
         with open(output_json_file, 'w') as open_file:
             json.dump(json_data, open_file, cls=DateTimeEncoder)
 
+    def get_json_schema_link(self):
+        sub_cli_version = get_sub_cli_version()
+        sub_cli_tags = get_sub_cli_github_tags()
+        if sub_cli_version in sub_cli_tags:
+            return f'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/refs/tags/v{sub_cli_version}/eva_sub_cli/etc/eva_schema.json'
+        else:
+            return 'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/main/eva_sub_cli/etc/eva_schema.json'
+
     def translate_header(self, title, header):
         if header in self.xlsx_conf[title].get(REQUIRED_HEADERS_KEY_NAME, {}):
             return self.xlsx_conf[title][REQUIRED_HEADERS_KEY_NAME][header]

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/nextflow/validation.nf

@@ -132,6 +132,9 @@ process trim_down_vcf {
 * Validate the VCF file format
 */
 process check_vcf_valid {
+
+    label 'long_time', 'med_mem'
+
     publishDir output_dir,
             overwrite: false,
             mode: "copy"
@@ -155,6 +158,9 @@ process check_vcf_valid {
 * Validate the VCF reference allele
 */
 process check_vcf_reference {
+
+    label 'long_time', 'med_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -178,6 +184,9 @@ process check_vcf_reference {
 }
 
 process generate_file_size_and_md5_digests {
+
+    label 'med_time', 'med_mem'
+
     input:
     path(vcf_file)
 
@@ -192,6 +201,9 @@ process generate_file_size_and_md5_digests {
 }
 
 process collect_file_size_and_md5 {
+
+    label 'med_time', 'med_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -210,6 +222,10 @@ process collect_file_size_and_md5 {
 
 
 process convert_xlsx_2_json {
+
+    label 'short_time', 'small_mem'
+
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -231,6 +247,9 @@ process convert_xlsx_2_json {
 }
 
 process metadata_json_validation {
+
+    label 'short_time', 'small_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -248,6 +267,9 @@ process metadata_json_validation {
 }
 
 process sample_name_concordance {
+
+    label 'default_time', 'default_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -268,6 +290,9 @@ process sample_name_concordance {
 
 
 process evidence_type_check {
+
+    label 'default_time', 'default_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -288,6 +313,9 @@ process evidence_type_check {
 
 
 process insdc_checker {
+
+    label 'long_time', 'default_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"
@@ -308,6 +336,9 @@ process insdc_checker {
 
 
 process metadata_semantic_check {
+
+    label 'default_time', 'default_mem'
+
     publishDir output_dir,
             overwrite: true,
             mode: "copy"

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/orchestrator.py

@@ -130,7 +130,10 @@ def get_project_and_vcf_fasta_mapping_from_metadata_json(metadata_json, mapping_
 
 def get_sub_cli_version():
     if version.parse(eva_sub_cli.__version__).is_devrelease:
-        major, minor, patch = map(int, version.parse(eva_sub_cli.__version__).base_version.split('.'))
+        version_values = [int(v) for v in version.parse(eva_sub_cli.__version__).base_version.split('.')]
+        major = version_values[0]
+        minor = version_values[1] if len(version_values) > 1 else 0
+        patch = version_values[2] if len(version_values) > 2 else 0
         if patch > 0:
             patch -= 1
         elif minor > 0:
@@ -160,7 +163,7 @@ def get_metadata_xlsx_template_link():
     sub_cli_version = get_sub_cli_version()
     sub_cli_tags = get_sub_cli_github_tags()
     if sub_cli_version in sub_cli_tags:
-        return f'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/v{sub_cli_version}/eva-sub-cli/eva_sub_cli/etc/EVA_Submission_template.xlsx'
+        return f'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/refs/tags/v{sub_cli_version}/eva_sub_cli/etc/EVA_Submission_template.xlsx'
     else:
         return 'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/main/eva_sub_cli/etc/EVA_Submission_template.xlsx'
 

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli/validators/validator.py

@@ -335,11 +335,8 @@ class Validator(AppLogger):
             with open(self._evidence_type_check_yaml) as open_yaml:
                 self.results['evidence_type_check'] = yaml.safe_load(open_yaml)
             self.results['evidence_type_check']['report_path'] = self._evidence_type_check_yaml
-
         self._update_metadata_with_evidence_type()
 
-
-
     def _collect_metadata_results(self):
         self.results['metadata_check'] = {}
         self._load_spreadsheet_conversion_errors()
@@ -478,7 +475,6 @@ class Validator(AppLogger):
         else:
             self.error(f'Cannot locate the metadata in JSON format in {os.path.join(self.output_dir, "metadata.json")}')
 
-
     def _update_metadata_with_evidence_type(self):
         if self.metadata_json_post_validation:
             metadata = EvaMetadataJson(self.metadata_json_post_validation)
@@ -487,8 +483,10 @@ class Validator(AppLogger):
                 if metadata.analyses:
                     for analysis in metadata.analyses:
                         analysis_alias = analysis['analysisAlias']
-                        analysis['evidenceType'] = self.results['evidence_type_check'][analysis_alias][
-                            'evidence_type'] if analysis_alias in self.results['evidence_type_check'] else None
+                        if (analysis_alias in self.results['evidence_type_check']
+                                and self.results['evidence_type_check'][analysis_alias]['evidence_type'] is not None):
+                            analysis['evidenceType'] = self.results['evidence_type_check'][analysis_alias][
+                                'evidence_type']
                         analysis_data.append(analysis)
                 else:
                     self.error('No analyses found in metadata')

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/eva_sub_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eva_sub_cli
-Version: 0.4.12
+Version: 0.4.14
 Summary: EBI EVA - validation and submission command line tool
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/tests/test_check_fasta_insdc.py

@@ -132,5 +132,5 @@ class TestFastaChecker(TestCase):
     @pytest.mark.integration
     def test_get_containing_assemblies(self):
         results = get_containing_assemblies('6ac8f815bf8e845bb3031b73f812c012')
-        assert len(results) == 14
+        assert len(results) == 15
         assert 'GCA_000001405.29' in results

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/tests/test_cli.py

@@ -9,6 +9,7 @@ from unittest.mock import patch, Mock
 
 from eva_sub_cli import orchestrator
 from eva_sub_cli.executables import cli
+from tests.test_utils import touch
 
 
 class TestCli(TestCase):
@@ -41,11 +42,17 @@ class TestCli(TestCase):
                 all_lines[0].endswith('[eva_sub_cli.orchestrator][DEBUG] test\n')
 
     def test_validate_args(self):
+        vcf_file = os.path.join(self.submission_dir,'test.vcf')
+        fasta_file = os.path.join(self.submission_dir, 'test.fasta')
+        json_file = os.path.join(self.submission_dir, 'test.json')
+        touch(vcf_file)
+        touch(fasta_file)
+        touch(json_file)
         cmd_args = [
             '--submission_dir', self.submission_dir,
-            '--vcf_files', 'test.vcf',
-            '--reference_fasta', 'test.fasta',
-            '--metadata_json', 'test.json',
+            '--vcf_files', vcf_file,
+            '--reference_fasta', fasta_file,
+            '--metadata_json', json_file,
             '--tasks', 'validate',
             '--executor', 'native',
             '--debug'

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/tests/test_orchestrator.py

@@ -306,7 +306,7 @@ class TestOrchestrator(unittest.TestCase):
                 patch('eva_sub_cli.orchestrator.get_sub_cli_github_tags') as sub_cli_tags:
             sub_cli_version.return_value = '1.1.6'
             sub_cli_tags.return_value = ['1.1.6']
-            assert get_metadata_xlsx_template_link() == 'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/v1.1.6/eva-sub-cli/eva_sub_cli/etc/EVA_Submission_template.xlsx'
+            assert get_metadata_xlsx_template_link() == 'https://raw.githubusercontent.com/EBIvariation/eva-sub-cli/refs/tags/v1.1.6/eva_sub_cli/etc/EVA_Submission_template.xlsx'
 
             sub_cli_version.return_value = '1.1.5'
             sub_cli_tags.return_value = ['1.1.6']

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/tests/test_validator.py

@@ -1,4 +1,5 @@
 import os.path
+import shutil
 from copy import deepcopy
 from unittest import TestCase
 
@@ -93,6 +94,7 @@ class TestValidator(TestCase):
     output_dir = os.path.join(resource_dir, 'validation_reports')
     mapping_file = os.path.join(output_dir, 'vcf_files_mapping.csv')
     metadata_xlsx_file = os.path.join(resource_dir, 'EVA_Submission_test.xlsx')
+    metadata_json_file = os.path.join(resource_dir, 'metadata_with_filename.json')
 
     def setUp(self) -> None:
         # create vcf mapping file
@@ -102,7 +104,10 @@ class TestValidator(TestCase):
                             [os.path.join(self.fasta_files, 'input_passed.fa')],
                             [os.path.join(self.assembly_reports, 'input_passed.txt')])
         self.validator = Validator(self.mapping_file, self.output_dir, metadata_xlsx=self.metadata_xlsx_file)
-        self.validator_json = Validator(self.mapping_file, self.output_dir)
+        self.validator_json = Validator(self.mapping_file, self.output_dir, metadata_json=self.metadata_json_file)
+        # Backup metadata json file so can restore after tests
+        self.backup_metadata_json = f'{self.metadata_json_file}.backup'
+        shutil.copy(self.metadata_json_file, self.backup_metadata_json)
 
     def tearDown(self) -> None:
         files_from_tests = [
@@ -115,6 +120,8 @@ class TestValidator(TestCase):
         for f in files_from_tests:
             if os.path.exists(f):
                 os.remove(f)
+        # Restore metadata json file
+        shutil.move(self.backup_metadata_json, self.metadata_json_file)
 
     def format_data_structure(self, source):
         if isinstance(source, dict):
@@ -329,3 +336,39 @@ class TestValidator(TestCase):
             os.remove(updated_metadata)
         # Reset metadata_json in case other tests need it
         self.validator_json.metadata_json = prev_metadata_json_value
+
+    def test__update_metadata_with_evidence_type_success(self):
+        self.validator_json.results['evidence_type_check'] = {
+            'AA': {
+                'errors': None,
+                'evidence_type': 'allele_frequency'
+            },
+            'report_path': '{resource_dir}/validation_reports/validation_output/other_validations/evidence_type_checker.yml'
+        }
+        self.validator_json._update_metadata_with_evidence_type()
+
+        # Analysis updated with evidence type
+        updated_metadata = EvaMetadataJson(self.validator_json.metadata_json_post_validation)
+        assert updated_metadata.analyses[0]['evidenceType'] == 'allele_frequency'
+
+    def test__update_metadata_with_evidence_type_failure(self):
+        self.validator_json.results['evidence_type_check'] = {
+            'AA': {
+                'errors': ['VCF file evidence type could not be determined'],
+                'evidence_type': None
+            },
+            'report_path': '{resource_dir}/validation_reports/validation_output/other_validations/evidence_type_checker.yml'
+        }
+        self.validator_json._update_metadata_with_evidence_type()
+
+        # Nothing added to analysis
+        updated_metadata = EvaMetadataJson(self.validator_json.metadata_json_post_validation)
+        assert 'evidenceType' not in updated_metadata.analyses[0]
+
+    def test__update_metadata_with_evidence_type_did_not_run(self):
+        self.validator_json.results['evidence_type_check'] = {}
+        self.validator_json._update_metadata_with_evidence_type()
+
+        # Nothing added to analysis
+        updated_metadata = EvaMetadataJson(self.validator_json.metadata_json_post_validation)
+        assert 'evidenceType' not in updated_metadata.analyses[0]

{eva_sub_cli-0.4.12 → eva_sub_cli-0.4.14}/tests/test_xlsx2json.py

@@ -14,7 +14,6 @@ class TestXlsReader(TestCase):
     resource_dir = os.path.join(os.path.dirname(__file__), 'resources')
     conf_filename = os.path.join(ETC_DIR, 'spreadsheet2json_conf.yaml')
     eva_schema = os.path.abspath(os.path.join(__file__, "../../eva_sub_cli/etc/eva_schema.json", ))
-    biosample_schema = os.path.abspath(os.path.join(__file__, "../../eva_sub_cli/etc/eva-biosamples.json", ))
 
     def tearDown(self):
         files_from_tests = [
@@ -55,22 +54,16 @@ class TestXlsReader(TestCase):
         with open(output_json) as open_file:
             json_data = json.load(open_file)
             # assert json file is created with expected data
-            assert sorted(json_data.keys()) == ['analysis', 'files', 'project', 'sample', 'submitterDetails']
+            assert sorted(json_data.keys()) == ['$schema', 'analysis', 'files', 'project', 'sample', 'submitterDetails']
+            json_data.pop('$schema', None)
             self.assertEqual(self.get_expected_json(), json_data)
 
         # assert json schema
         with open(self.eva_schema) as eva_schema_file:
             eva_json_schema = json.load(eva_schema_file)
-        with open(self.biosample_schema) as biosample_schema_file:
-            biosample_json_schema = json.load(biosample_schema_file)
-
-        # assert created json file sample field conforms to eva-biosamples schema
-        jsonschema.validate(json_data['sample'][3]['bioSampleObject'], biosample_json_schema)
 
         # assert created json file conform to eva_schema
-        resolver = jsonschema.RefResolver.from_schema(eva_json_schema)
-        resolver.store['eva-biosamples.json'] = biosample_json_schema
-        jsonschema.validate(json_data, eva_json_schema, resolver=resolver)
+        jsonschema.validate(json_data, eva_json_schema)
 
     def test_create_xls_template(self):
         metadata_file = os.path.join(self.resource_dir, 'metadata_not_existing.xlsx')
@@ -94,7 +87,7 @@ class TestXlsReader(TestCase):
         assert os.path.exists(output_json)
         with open(output_json) as open_file:
             json_data = json.load(open_file)
-            assert sorted(json_data.keys()) == ['analysis', 'files', 'project', 'sample', 'submitterDetails']
+            assert sorted(json_data.keys()) == ['$schema', 'analysis', 'files', 'project', 'sample', 'submitterDetails']
             # required field taxId is missing
             assert 'taxId' not in json_data['project']
             # novel sample is missing scientific name in characteristics and sample name
@@ -105,10 +98,6 @@ class TestXlsReader(TestCase):
     def test_json_validation_fails_for_large_project_title_and_project_description(self) -> None:
         with open(self.eva_schema) as eva_schema_file:
             eva_json_schema = json.load(eva_schema_file)
-        with open(self.biosample_schema) as biosample_schema_file:
-            biosample_json_schema = json.load(biosample_schema_file)
-        resolver = jsonschema.RefResolver.from_schema(eva_json_schema)
-        resolver.store['eva-biosamples.json'] = biosample_json_schema
 
         # test fails for title
         json_data = self.get_expected_json()
@@ -116,7 +105,7 @@ class TestXlsReader(TestCase):
 
         assert len(json_data['project']['title']) == 600
         with self.assertRaises(jsonschema.ValidationError):
-            jsonschema.validate(json_data, eva_json_schema, resolver=resolver)
+            jsonschema.validate(json_data, eva_json_schema)
 
         # test fails for description
         json_data = self.get_expected_json()
@@ -124,7 +113,7 @@ class TestXlsReader(TestCase):
 
         assert len(json_data['project']['description']) == 6000
         with self.assertRaises(jsonschema.ValidationError):
-            jsonschema.validate(json_data, eva_json_schema, resolver=resolver)
+            jsonschema.validate(json_data, eva_json_schema)
 
     def build_large_string_of_length(self, length):
         return "A" * length
@@ -220,20 +209,23 @@ class TestXlsReader(TestCase):
                             "taxId": [
                                 {"text": "9447"}
                             ],
-                            "scientificName": [
+                            "scientific name": [
                                 {"text": "Lemur catta"}
                             ],
                             "sex": [
                                 {"text": "Female"}
                             ],
-                            "tissueType": [
+                            "tissue_type": [
                                 {"text": "skin"}
                             ],
                             "species": [
                                 {"text": "Lemur catta"}
                             ],
-                            'collectionDate': [
+                            'collection date': [
                                 {'text': '2021-03-12'}
+                            ],
+                            'geographic location (country and/or sea)': [
+                                {'text': 'Afghanistan'}
                             ]
                         }
                     }

eva_sub_cli-0.4.12/docs/installation.md

@@ -1,49 +0,0 @@
-# Installation
-
-There are currently three ways to install and run eva-sub-cli:
-- Using conda
-- From source using Docker
-- From source natively (i.e. installing dependencies yourself)
-
-We encourage users to install the most recent version, as we are constantly improving the tool. You can check that you
-have the latest version installed by running `eva-sub-cli.py --version` and comparing against the
-[release page](https://github.com/EBIvariation/eva-sub-cli/releases).
-
-## 1. Conda
-
-The most straightforward way to install eva-sub-cli and its dependencies is through conda.
-For instance, the following commands install eva-sub-cli in a new environment called `eva`, activate the environment, and print
-the help message:
-```bash
-conda create -n eva -c conda-forge -c bioconda eva-sub-cli
-conda activate eva
-eva-sub-cli.py --help
-````
-
-To upgrade to the newest version, run `conda update eva-sub-cli`.
-
-## 2. From source using Docker
-
-Docker provides an easy way to run eva-sub-cli without installing dependencies separately.
-This method requires just Python 3.8+ and [Docker](https://docs.docker.com/engine/install/) to be installed.
-Then you can install the most recent version from [PyPI](https://pypi.org/project/eva-sub-cli/) in a virtual environment:
-```bash
-pip install eva-sub-cli
-```
-
-To verify that the cli tool is installed correctly, run the following command, and you should see the help message displayed: 
-```bash
-eva-sub-cli.py --help
-```
-
-To upgrade to the newest version, run `pip install --upgrade eva-sub-cli`.
-
-## 3. From source natively
-
-This installation method requires the following:
-* Python 3.8+
-* [Nextflow](https://www.nextflow.io/docs/latest/getstarted.html) 21.10+
-* [biovalidator](https://github.com/elixir-europe/biovalidator) 2.1.0+
-* [vcf-validator](https://github.com/EBIvariation/vcf-validator) 0.9.7+
-
-Install each of these and ensure they are included in your PATH. Then install the latest eva-sub-cli release as previously described.