verdy 0.1.0__tar.gz

verdy-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jovana Dobreva
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

verdy-0.1.0/PKG-INFO

@@ -0,0 +1,154 @@
+Metadata-Version: 2.4
+Name: verdy
+Version: 0.1.0
+Summary: VERDY: match your own terms (species, habitats, parameters, measurements) to the ontologies you choose: ENVO, NCBITaxon, GBIF, PATO, CF, Wikidata, any OLS4 ontology.
+Author: Jovana Dobreva
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/dobrevajovana/verdy
+Project-URL: Source, https://github.com/dobrevajovana/verdy
+Project-URL: Issues, https://github.com/dobrevajovana/verdy/issues
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.31
+Provides-Extra: excel
+Requires-Dist: pandas>=2.0; extra == "excel"
+Requires-Dist: openpyxl>=3.1; extra == "excel"
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Dynamic: license-file
+
+<p align="center"><img src="https://raw.githubusercontent.com/dobrevajovana/verdy/main/logo.png" width="200" alt="VERDY logo: a green forest spirit with a flower crown, braided leafy hair and berry nodes linked by vines"></p>
+
+# VERDY
+
+[![Python package](https://github.com/dobrevajovana/verdy/actions/workflows/python-package.yml/badge.svg)](https://github.com/dobrevajovana/verdy/actions/workflows/python-package.yml)
+
+*Part of **VERDANT**: **V**ocabulary **E**nrichment & **R**esolution of **D**iversity **A**nnotations via **N**eural **T**echnology
+(BioHackathon Europe 2026).*
+
+**Meet VERDY, your warrior for ontology matching.** Give it your ENA IDs or free-text terms, choose the ontologies, and it predicts the matching terms in each.
+
+1. **Input:** a file (or Python list) of ENA IDs and/or free text. The IDs can be samples, runs or projects. The free text can be species, habitats, parameters or measurement names.
+2. **ENA lookup:** each ENA ID is looked up in [ENA](https://www.ebi.ac.uk/ena/portal/api) for its sample's organism and environment fields (`scientific_name`, `broad_scale_environmental_context`, `local_environmental_context`, `environmental_medium`, ...). Free text is used as it is.
+3. **Search:** every name is searched in each ontology you choose (for example `natura2000`, `gold`, `envo`).
+4. **Output:** a file with the exact match, or the most closely related terms, per name and ontology.
+
+```python
+import verdy
+
+matches = verdy.match("my_input.csv", ontologies=["natura2000", "gold", "envo", "ncbitaxon"])
+verdy.save(matches, "my_input.matches.csv")      # .csv  .tsv  .sssom.tsv  .json  .xlsx
+```
+
+```
+verdy my_input.csv -o natura2000,gold,envo --out my_input.matches.csv
+```
+
+## Install
+
+```
+pip install git+https://github.com/dobrevajovana/verdy.git
+pip install "verdy[excel] @ git+https://github.com/dobrevajovana/verdy.git"    # to read and write Excel
+```
+
+Python 3.10 or newer; the only dependency is `requests`.
+
+## Input
+
+`verdy.match` takes a CSV / TSV / Excel file, a text file with one value per line, a Python list or a pandas DataFrame.
+The column is found by its header (`name`, `term`, `parameter`, `species`, ..., or `id`, `accession`, `sample_accession`, `run_accession`)
+or given with `column="..."`. One file can mix both kinds of value:
+
+| value | what happens |
+|---|---|
+| `SAMD00132352`, `SAMEA...`, `SAMN...`, `ERS...` | an ENA sample: its fields are looked up in ENA and each value is searched |
+| `ERR9966617`, `SRR...`, `ERX...` | a run or experiment: its sample, as above |
+| `PRJEB54740`, `PRJNA...` | a project: all its samples, each row under the sample's own accession |
+| anything else | free text, searched as it is (units in brackets, underscores and a leading code are set aside) |
+
+`fields=` chooses other ENA sample fields to search (see ENA's `returnFields?result=sample`).
+After a run, `verdy.match.unresolved` lists the IDs ENA did not know.
+
+## The ontologies
+
+`verdy.ONTOLOGIES` (or `verdy --list`):
+
+| ID | What | For |
+|---|---|---|
+| `natura2000` | Natura 2000 habitat types and species (EEA Article 17 lists, fetched live) | habitats by name or code, protected species |
+| `envo` | Environment Ontology | habitats, environments, materials, environmental measurements |
+| `gold` | GOLD ecosystem classification | sample ecosystems |
+| `ncbitaxon` | NCBI Taxonomy | organisms |
+| `gbif` | GBIF Backbone Taxonomy | organisms |
+| `pato` | Phenotype And Trait Ontology | qualities: temperature, salinity, humidity, ... |
+| `oba` | Ontology of Biological Attributes | traits and measurable attributes |
+| `cf` | CF Standard Names | climate, weather and ocean variables |
+| `uo` | Units of measurement | units |
+| `chebi` | Chemical Entities of Biological Interest | chemicals |
+| `obi` | Ontology for Biomedical Investigations | assays, instruments, protocols |
+| `uberon`, `po`, `go` | anatomy, plants, biological processes | |
+| `wikidata` | Wikidata | anything with a name |
+
+**Any other ontology in [OLS4](https://www.ebi.ac.uk/ols4/ontologies) works as well:** pass its ID
+(for example `foodon`, `pco` or `ms`).
+
+## What comes out
+
+One row per match:
+
+| column | |
+|---|---|
+| `term_id`, `term`, `category` | your term |
+| `match_type` | `skos:exactMatch`: a label or synonym is the term (plurals and `sp.` aside) · `skos:closeMatch`: most words shared · `skos:relatedMatch`: some words shared |
+| `ontology`, `match_curie`, `match_id`, `match_label` | the matched term |
+| `confidence` | 1.0 exact as written, 0.95 exact up to plurals, then by word overlap |
+| `word_overlap`, `rank` | how many words they share (0..1), and the place among this term's matches |
+| `definition` | the term's definition (units first, for CF) |
+| `searched_for` | the term after cleaning |
+
+How the search works:
+* At most `per_ontology` matches (default 3) per term and ontology. When there is an exact match, only that one is given.
+* In the taxonomies (NCBITaxon, GBIF), only exact names count, because a near name is another taxon.
+* `verdy.unmatched(terms, matches, ontologies)` lists, per ontology, the terms it found nothing for.
+
+Every answer (ENA, OLS4, GBIF, Wikidata, the CF table, the Natura 2000 lists) is cached in `~/.cache/verdy` (or `VERDY_CACHE`), so a second run makes no requests.
+
+## Example
+
+`python examples/predict.py` runs [examples/input.csv](examples/input.csv), which has three ENA accessions and four free-text terms, against
+Natura 2000, GOLD, ENVO, NCBITaxon, PATO and CF. It writes [examples/input.matches.csv](examples/input.matches.csv). Part of the result:
+
+| input | name searched (from) | ontology | match |
+|---|---|---|---|
+| SAMD00132352 | urban biome (broad_scale_environmental_context) | ENVO | exactMatch ENVO:01000249 urban biome |
+| SAMD00132352 | air (local_environmental_context, environmental_medium) | ENVO · GOLD | exactMatch ENVO:00002005 air · GOLD:Air |
+| SAMD00132352 | air metagenome (scientific_name) | NCBITaxon | exactMatch NCBITaxon:655179 |
+| ERR9966617 | temperate forest | ENVO | exactMatch ENVO:01001805 temperate forest |
+| ERR9966617 | mountain | ENVO | exactMatch ENVO:00000081 mountain |
+| SAMEA110429224 | Human (host_scientific_name) | NCBITaxon | exactMatch NCBITaxon:9606 Homo sapiens |
+| Lynx lynx | | Natura 2000 · NCBITaxon | exactMatch n2k:1361 · NCBITaxon:13125 |
+| Atlantic salt meadows | | Natura 2000 | exactMatch n2k:1330 |
+| air_temperature | | CF · ENVO | exactMatch CF:air_temperature · ENVO:09200001 temperature of air |
+| salinity | | PATO | exactMatch PATO:0085001 |
+
+## One term at a time
+
+```python
+verdy.search("cloud liquid water content", "cf")    # the source's best terms, best first
+verdy.resolve("SAMD00132352")                      # what ENA says about an ID
+verdy.read_terms("my_input.csv")                    # what VERDY read from your file
+verdy.to_dataframe(matches)                         # pandas
+```
+
+## Tests
+
+```
+pip install -e ".[test]" && python -m pytest -q
+```
+
+The tests stub every web service, so they run offline.
+
+## Licence
+
+MIT, see [LICENSE](LICENSE).

verdy-0.1.0/README.md

@@ -0,0 +1,134 @@
+<p align="center"><img src="https://raw.githubusercontent.com/dobrevajovana/verdy/main/logo.png" width="200" alt="VERDY logo: a green forest spirit with a flower crown, braided leafy hair and berry nodes linked by vines"></p>
+
+# VERDY
+
+[![Python package](https://github.com/dobrevajovana/verdy/actions/workflows/python-package.yml/badge.svg)](https://github.com/dobrevajovana/verdy/actions/workflows/python-package.yml)
+
+*Part of **VERDANT**: **V**ocabulary **E**nrichment & **R**esolution of **D**iversity **A**nnotations via **N**eural **T**echnology
+(BioHackathon Europe 2026).*
+
+**Meet VERDY, your warrior for ontology matching.** Give it your ENA IDs or free-text terms, choose the ontologies, and it predicts the matching terms in each.
+
+1. **Input:** a file (or Python list) of ENA IDs and/or free text. The IDs can be samples, runs or projects. The free text can be species, habitats, parameters or measurement names.
+2. **ENA lookup:** each ENA ID is looked up in [ENA](https://www.ebi.ac.uk/ena/portal/api) for its sample's organism and environment fields (`scientific_name`, `broad_scale_environmental_context`, `local_environmental_context`, `environmental_medium`, ...). Free text is used as it is.
+3. **Search:** every name is searched in each ontology you choose (for example `natura2000`, `gold`, `envo`).
+4. **Output:** a file with the exact match, or the most closely related terms, per name and ontology.
+
+```python
+import verdy
+
+matches = verdy.match("my_input.csv", ontologies=["natura2000", "gold", "envo", "ncbitaxon"])
+verdy.save(matches, "my_input.matches.csv")      # .csv  .tsv  .sssom.tsv  .json  .xlsx
+```
+
+```
+verdy my_input.csv -o natura2000,gold,envo --out my_input.matches.csv
+```
+
+## Install
+
+```
+pip install git+https://github.com/dobrevajovana/verdy.git
+pip install "verdy[excel] @ git+https://github.com/dobrevajovana/verdy.git"    # to read and write Excel
+```
+
+Python 3.10 or newer; the only dependency is `requests`.
+
+## Input
+
+`verdy.match` takes a CSV / TSV / Excel file, a text file with one value per line, a Python list or a pandas DataFrame.
+The column is found by its header (`name`, `term`, `parameter`, `species`, ..., or `id`, `accession`, `sample_accession`, `run_accession`)
+or given with `column="..."`. One file can mix both kinds of value:
+
+| value | what happens |
+|---|---|
+| `SAMD00132352`, `SAMEA...`, `SAMN...`, `ERS...` | an ENA sample: its fields are looked up in ENA and each value is searched |
+| `ERR9966617`, `SRR...`, `ERX...` | a run or experiment: its sample, as above |
+| `PRJEB54740`, `PRJNA...` | a project: all its samples, each row under the sample's own accession |
+| anything else | free text, searched as it is (units in brackets, underscores and a leading code are set aside) |
+
+`fields=` chooses other ENA sample fields to search (see ENA's `returnFields?result=sample`).
+After a run, `verdy.match.unresolved` lists the IDs ENA did not know.
+
+## The ontologies
+
+`verdy.ONTOLOGIES` (or `verdy --list`):
+
+| ID | What | For |
+|---|---|---|
+| `natura2000` | Natura 2000 habitat types and species (EEA Article 17 lists, fetched live) | habitats by name or code, protected species |
+| `envo` | Environment Ontology | habitats, environments, materials, environmental measurements |
+| `gold` | GOLD ecosystem classification | sample ecosystems |
+| `ncbitaxon` | NCBI Taxonomy | organisms |
+| `gbif` | GBIF Backbone Taxonomy | organisms |
+| `pato` | Phenotype And Trait Ontology | qualities: temperature, salinity, humidity, ... |
+| `oba` | Ontology of Biological Attributes | traits and measurable attributes |
+| `cf` | CF Standard Names | climate, weather and ocean variables |
+| `uo` | Units of measurement | units |
+| `chebi` | Chemical Entities of Biological Interest | chemicals |
+| `obi` | Ontology for Biomedical Investigations | assays, instruments, protocols |
+| `uberon`, `po`, `go` | anatomy, plants, biological processes | |
+| `wikidata` | Wikidata | anything with a name |
+
+**Any other ontology in [OLS4](https://www.ebi.ac.uk/ols4/ontologies) works as well:** pass its ID
+(for example `foodon`, `pco` or `ms`).
+
+## What comes out
+
+One row per match:
+
+| column | |
+|---|---|
+| `term_id`, `term`, `category` | your term |
+| `match_type` | `skos:exactMatch`: a label or synonym is the term (plurals and `sp.` aside) · `skos:closeMatch`: most words shared · `skos:relatedMatch`: some words shared |
+| `ontology`, `match_curie`, `match_id`, `match_label` | the matched term |
+| `confidence` | 1.0 exact as written, 0.95 exact up to plurals, then by word overlap |
+| `word_overlap`, `rank` | how many words they share (0..1), and the place among this term's matches |
+| `definition` | the term's definition (units first, for CF) |
+| `searched_for` | the term after cleaning |
+
+How the search works:
+* At most `per_ontology` matches (default 3) per term and ontology. When there is an exact match, only that one is given.
+* In the taxonomies (NCBITaxon, GBIF), only exact names count, because a near name is another taxon.
+* `verdy.unmatched(terms, matches, ontologies)` lists, per ontology, the terms it found nothing for.
+
+Every answer (ENA, OLS4, GBIF, Wikidata, the CF table, the Natura 2000 lists) is cached in `~/.cache/verdy` (or `VERDY_CACHE`), so a second run makes no requests.
+
+## Example
+
+`python examples/predict.py` runs [examples/input.csv](examples/input.csv), which has three ENA accessions and four free-text terms, against
+Natura 2000, GOLD, ENVO, NCBITaxon, PATO and CF. It writes [examples/input.matches.csv](examples/input.matches.csv). Part of the result:
+
+| input | name searched (from) | ontology | match |
+|---|---|---|---|
+| SAMD00132352 | urban biome (broad_scale_environmental_context) | ENVO | exactMatch ENVO:01000249 urban biome |
+| SAMD00132352 | air (local_environmental_context, environmental_medium) | ENVO · GOLD | exactMatch ENVO:00002005 air · GOLD:Air |
+| SAMD00132352 | air metagenome (scientific_name) | NCBITaxon | exactMatch NCBITaxon:655179 |
+| ERR9966617 | temperate forest | ENVO | exactMatch ENVO:01001805 temperate forest |
+| ERR9966617 | mountain | ENVO | exactMatch ENVO:00000081 mountain |
+| SAMEA110429224 | Human (host_scientific_name) | NCBITaxon | exactMatch NCBITaxon:9606 Homo sapiens |
+| Lynx lynx | | Natura 2000 · NCBITaxon | exactMatch n2k:1361 · NCBITaxon:13125 |
+| Atlantic salt meadows | | Natura 2000 | exactMatch n2k:1330 |
+| air_temperature | | CF · ENVO | exactMatch CF:air_temperature · ENVO:09200001 temperature of air |
+| salinity | | PATO | exactMatch PATO:0085001 |
+
+## One term at a time
+
+```python
+verdy.search("cloud liquid water content", "cf")    # the source's best terms, best first
+verdy.resolve("SAMD00132352")                      # what ENA says about an ID
+verdy.read_terms("my_input.csv")                    # what VERDY read from your file
+verdy.to_dataframe(matches)                         # pandas
+```
+
+## Tests
+
+```
+pip install -e ".[test]" && python -m pytest -q
+```
+
+The tests stub every web service, so they run offline.
+
+## Licence
+
+MIT, see [LICENSE](LICENSE).

verdy-0.1.0/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "verdy"
+version = "0.1.0"
+description = "VERDY: match your own terms (species, habitats, parameters, measurements) to the ontologies you choose: ENVO, NCBITaxon, GBIF, PATO, CF, Wikidata, any OLS4 ontology."
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = ["requests>=2.31"]
+authors = [{ name = "Jovana Dobreva" }]
+license = "MIT"
+license-files = ["LICENSE"]
+
+[project.urls]
+Homepage = "https://github.com/dobrevajovana/verdy"
+Source = "https://github.com/dobrevajovana/verdy"
+Issues = "https://github.com/dobrevajovana/verdy/issues"
+
+[project.optional-dependencies]
+excel = ["pandas>=2.0", "openpyxl>=3.1"]
+test = ["pytest>=7"]
+
+[project.scripts]
+verdy = "verdy.__main__:main"
+
+[tool.setuptools]
+packages = ["verdy"]

verdy-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

verdy-0.1.0/tests/test_match.py

@@ -0,0 +1,194 @@
+"""verdy.match with every web service stubbed.
+
+Run:  python -m pytest -q
+"""
+
+import csv
+import json
+
+import pytest
+
+import verdy
+from verdy import sources
+
+ENVO = "http://purl.obolibrary.org/obo/ENVO_"
+TAXON = "http://purl.obolibrary.org/obo/NCBITaxon_"
+PATO = "http://purl.obolibrary.org/obo/PATO_"
+CF = b"""<?xml version="1.0"?><standard_name_table>
+<entry id="air_temperature"><canonical_units>K</canonical_units><description>Air temperature is the bulk temperature of the air.</description></entry>
+<entry id="wind_speed"><canonical_units>m s-1</canonical_units><description>Speed is the magnitude of velocity.</description></entry>
+<alias id="surface_air_temperature"><entry_id>air_temperature</entry_id></alias>
+</standard_name_table>"""
+
+
+@pytest.fixture
+def services(monkeypatch, tmp_path):
+    """OLS4 (ENVO, NCBITaxon, PATO), GBIF, Wikidata and the CF table, answering from these tables."""
+    monkeypatch.setattr(sources, "CACHE", tmp_path / "cache")
+    monkeypatch.setattr(sources, "_cf_terms", None)
+    (tmp_path / "cache").mkdir()
+    (tmp_path / "cache" / "cf-standard-name-table.xml").write_bytes(CF)
+    ols = {("envo", "estuary"): [{"iri": ENVO + "00000045", "label": "estuary", "obo_id": "ENVO:00000045"}],
+           ("envo", "lagoon"): [{"iri": ENVO + "00000038", "label": "lagoon", "obo_id": "ENVO:00000038"}],
+           ("envo", "coastal"): [{"iri": ENVO + "1", "label": "coastal water", "obo_id": "ENVO:1"}],
+           ("ncbitaxon", "lynx lynx"): [{"iri": TAXON + "13125", "label": "Lynx lynx", "obo_id": "NCBITaxon:13125"}],
+           ("ncbitaxon", "estuaries"): [{"iri": TAXON + "1", "label": "estuaries bacterium"}],
+           ("pato", "temperature"): [{"iri": PATO + "0000146", "label": "temperature", "obo_id": "PATO:0000146"}],
+           ("envo", "urban biome"): [{"iri": ENVO + "01000249", "label": "urban biome", "obo_id": "ENVO:01000249",
+                                      "is_defining_ontology": True}],
+           ("envo", "air"): [{"iri": ENVO + "00002005", "label": "air", "obo_id": "ENVO:00002005", "is_defining_ontology": True}],
+           ("gold", "air"): [{"iri": "https://w3id.org/gold.path/3962", "label": "Environmental > Air > Outdoor Air",
+                              "is_defining_ontology": False},
+                             {"iri": "https://w3id.org/gold.vocab/Air", "label": "Air", "is_defining_ontology": False}]}
+    calls = []
+
+    def fake(url, **p):
+        calls.append(url)
+        if url.endswith("/portal/api/search"):                         # ENA samples
+            acc = p["query"].split('"')[1]
+            return [{"sample_accession": "SAMD1", "scientific_name": "air metagenome", "broad_scale_environmental_context":
+                     "urban biome", "local_environmental_context": "air", "environmental_medium": "air",
+                     "isolation_source": "missing"}] if acc in ("SAMD1", "ERS1") else []
+        if url.endswith("/portal/api/filereport"):                     # ENA runs and projects
+            return {"ERR1": [{"sample_accession": "SAMD1"}], "PRJEB1": [{"sample_accession": "SAMD1"},
+                    {"sample_accession": "SAMD1"}]}.get(p["accession"], [])
+        if "/ontologies/" in url:
+            oid = url.rsplit("/", 1)[1]
+            return {"config": {"title": oid, "preferredPrefix": {"ncbitaxon": "NCBITaxon"}.get(oid, oid.upper())}} \
+                if oid in ("envo", "ncbitaxon", "pato", "gold") else {}
+        if url.endswith("/search"):
+            return {"response": {"docs": ols.get((p["ontology"], p["q"].lower()), [])}}
+        if "species/match" in url:
+            return {"usageKey": 2435240, "canonicalName": "Lynx lynx", "matchType": "EXACT"} if p["name"] == "Lynx lynx" \
+                else {"matchType": "NONE"}
+        if "MapServer" in url:                                          # Natura 2000 (Article 17)
+            if "/5/" in url:
+                return {"features": [{"attributes": {"reference_habitatlist.habitat_code": "1130",
+                                                     "reference_habitatlist.label": "Estuaries"}}]}
+            return {"features": [{"attributes": {"reference_specieslist.species_code": "1361",
+                                                 "reference_specieslist.species_name": "Lynx lynx",
+                                                 "reference_specieslist.taxonomical_group": "Mammals"}}]}
+        if "wikidata" in url:
+            return {"search": [{"id": "Q43375", "label": "Eurasian lynx", "aliases": ["Lynx lynx"]}]} if p["search"] == "Lynx lynx" \
+                else {"search": [{"id": "Q9", "label": "some moth", "aliases": ["coastal lagoons moth"]}]}
+        return {}
+    monkeypatch.setattr(sources, "_json", fake)
+    return calls
+
+
+def test_read_terms_from_many_inputs(tmp_path):
+    (tmp_path / "a.csv").write_text("id,Parameter,group\nx1,air temperature (°C),parameter\nx2,Estuaries,habitat\nx2,again,habitat\n")
+    (tmp_path / "b.txt").write_text("Lynx lynx\n\nEstuaries\n")
+    (tmp_path / "c.tsv").write_text("code\tlabel\n1130\tEstuaries\n")
+    assert verdy.read_terms(tmp_path / "a.csv") == [
+        {"id": "x1", "label": "air temperature (°C)", "category": "parameter"}, {"id": "x2", "label": "Estuaries", "category": "habitat"}]
+    assert [t["label"] for t in verdy.read_terms(tmp_path / "b.txt")] == ["Lynx lynx", "Estuaries"]
+    assert verdy.read_terms(tmp_path / "c.tsv")[0] == {"id": "1130", "label": "Estuaries", "category": None}
+    assert [t["id"] for t in verdy.read_terms(["Lynx lynx", {"name": "Coastal lagoons"}])] == ["lynx_lynx", "coastal_lagoons"]
+    with pytest.raises(ValueError):
+        verdy.read_terms(tmp_path / "a.csv", column="nope")
+
+
+def test_only_the_chosen_ontologies(services):
+    ms = verdy.match(["Lynx lynx", "Estuaries", "Coastal lagoons"], ["envo"])
+    assert {m["ontology"] for m in ms} == {"ENVO"}
+    by = {(m["term"], m["match_label"]): m for m in ms}
+    assert by[("Estuaries", "estuary")]["match_type"] == "skos:exactMatch"            # plural aside
+    assert by[("Coastal lagoons", "lagoon")]["match_type"] == "skos:relatedMatch"
+    assert not any(m["term"] == "Lynx lynx" for m in ms)
+    assert not any("wikidata" in c or "gbif" in c for c in services)                  # not chosen, not asked
+
+
+def test_taxonomies_take_exact_names_only(services):
+    ms = verdy.match(["Lynx lynx", "Estuaries"], "ncbitaxon, gbif, wikidata")
+    got = {(m["term"], m["ontology"]): m for m in ms}
+    assert got[("Lynx lynx", "NCBITaxon")]["match_curie"] == "NCBITaxon:13125"
+    assert got[("Lynx lynx", "GBIF")]["match_curie"] == "GBIF:2435240"
+    assert got[("Lynx lynx", "Wikidata")]["match_type"] == "skos:exactMatch"          # by its alias
+    assert ("Estuaries", "NCBITaxon") not in got                                     # 'estuaries bacterium' is not it
+    assert ("Estuaries", "Wikidata") not in got                                      # an alias alone is no near match
+    assert verdy.unmatched(["Lynx lynx", "Estuaries"], ms, "ncbitaxon,gbif") == {"ncbitaxon": ["Estuaries"], "gbif": ["Estuaries"]}
+
+
+def test_parameters_in_cf_and_pato(services):
+    ms = verdy.match(["air temperature [K]", "wind speed", "surface air temperature"], ["cf", "pato"])
+    got = {(m["term"], m["ontology"], m["match_curie"]): m["match_type"] for m in ms}
+    assert got[("air temperature [K]", "CF", "CF:air_temperature")] == "skos:exactMatch"      # units in brackets aside
+    assert got[("surface air temperature", "CF", "CF:air_temperature")] == "skos:exactMatch"  # a CF alias
+    assert got[("wind speed", "CF", "CF:wind_speed")] == "skos:exactMatch"
+    assert got[("air temperature [K]", "PATO", "PATO:0000146")] == "skos:relatedMatch"
+    assert next(m for m in ms if m["match_curie"] == "CF:air_temperature")["definition"].startswith("[K]")
+
+
+def test_leading_codes_units_and_underscores():
+    from verdy.sources import query_text
+    assert query_text("1330 Atlantic salt meadows") == "Atlantic salt meadows"
+    assert query_text("air temperature [K]") == "air temperature" and query_text("air_temperature") == "air temperature"
+    assert query_text("16S rRNA gene") == "16S rRNA gene"
+
+
+def test_unknown_or_no_ontology(services):
+    with pytest.raises(ValueError, match="nosuch"):
+        verdy.match(["Lynx lynx"], ["nosuch"])
+    with pytest.raises(ValueError):
+        verdy.match(["Lynx lynx"], [])
+
+
+def test_outputs(services, tmp_path):
+    ms = verdy.match(["Lynx lynx", "Estuaries"], ["envo", "ncbitaxon"])
+    for f in ("m.csv", "m.tsv", "m.json", "m.sssom.tsv"):
+        verdy.save(ms, tmp_path / f)
+    rows = list(csv.DictReader(open(tmp_path / "m.csv")))
+    assert {r["match_curie"] for r in rows} == {"ENVO:00000045", "NCBITaxon:13125"}
+    assert json.loads((tmp_path / "m.json").read_text()) == ms
+    sssom = (tmp_path / "m.sssom.tsv").read_text()
+    assert "semapv:LexicalMatching" in sssom and "#   ENVO: http://purl.obolibrary.org/obo/ENVO_" in sssom
+
+
+def test_cache_makes_no_second_request(services):
+    verdy.match(["Estuaries"], ["envo"])
+    n = len(services)
+    verdy.match(["Estuaries"], ["envo"])
+    assert len(services) == n
+
+
+def test_command_line(services, tmp_path, capsys):
+    from verdy.__main__ import main
+    (tmp_path / "n.csv").write_text("name\nLynx lynx\nEstuaries\n")
+    assert main(["--list"]) == 0 and "ncbitaxon" in capsys.readouterr().out
+    assert main([str(tmp_path / "n.csv"), "-o", "envo,ncbitaxon", "--out", str(tmp_path / "o.csv")]) == 0
+    err = capsys.readouterr().err
+    assert "2 matches for 2 of 2 inputs" in err and "nothing in envo for: Lynx lynx" in err
+    assert (tmp_path / "o.csv").exists()
+
+
+def test_ids_are_looked_up_in_ena_and_the_ontologies(services):
+    import importlib
+    r = importlib.import_module("verdy.resolve")
+    assert [r.kind(x) for x in ("SAMD1", "ERS1", "ERR1", "PRJEB1", "ENVO:00000045", "air")] == \
+        ["sample", "sample", "run", "project", "name", "name"]
+    assert r.resolve("SAMD1") == [("scientific_name", "air metagenome"), ("broad_scale_environmental_context", "urban biome"),
+                                  ("local_environmental_context", "air"), ("environmental_medium", "air")]    # 'missing' left out
+    assert r.resolve("ERR1") == r.resolve("SAMD1")
+    assert r.resolve("air temperature") == [("name", "air temperature")]
+    assert r.resolve("SAMD999") == []
+
+
+def test_ids_in_predictions_out(services, tmp_path):
+    (tmp_path / "ids.csv").write_text("sample_accession\nSAMD1\nEstuaries\nPRJEB1\nSAMD999\n")
+    ms = verdy.match(tmp_path / "ids.csv", ["envo", "gold", "natura2000"])
+    rows = {(m["term_id"], m["term"], m["ontology"], m["match_curie"] or m["match_id"]): m for m in ms}
+    air = rows[("SAMD1", "air", "ENVO", "ENVO:00002005")]
+    assert air["category"] == "local_environmental_context|environmental_medium" and air["match_type"] == "skos:exactMatch"
+    assert ("SAMD1", "urban biome", "ENVO", "ENVO:01000249") in rows
+    assert any(k[0] == "SAMD1" and k[2] == "GOLD" and "Air" in rows[k]["match_label"] for k in rows)   # GOLD kept
+    assert rows[("estuaries", "Estuaries", "Natura2000", "n2k:1130")]["match_type"] == "skos:exactMatch"
+    assert verdy.match.unresolved == ["SAMD999"]
+    assert not any(m["term_id"] == "PRJEB1" for m in ms)            # a project's rows carry its samples' IDs
+
+
+def test_free_text_and_ena_ids_together(services):
+    ms = verdy.match(["SAMD1", "Estuaries"], ["envo"])
+    assert {(m["term_id"], m["term"]) for m in ms} >= {("SAMD1", "air"), ("SAMD1", "urban biome"), ("estuaries", "Estuaries")}
+    sssom = verdy.to_sssom(ms)
+    assert "biosample:SAMD1\t" in sssom and "term:estuaries" in sssom

verdy-0.1.0/verdy/__init__.py

@@ -0,0 +1,34 @@
+"""VERDY: predict matches for your ENA IDs or free-text terms in the ontologies you choose.
+
+Part of VERDANT: Vocabulary Enrichment & Resolution of Diversity Annotations via Neural Technology.
+
+Input (a CSV / TSV / Excel file, a text file or a Python list) holds ENA IDs (samples,
+runs, projects) and/or free text (species, habitats, parameters, measurements). ENA IDs
+are looked up in ENA for their organism and environment fields; every name is searched in
+each ontology you choose; out come its exact and most related terms.
+
+    import verdy
+
+    matches = verdy.match("my_input.csv", ontologies=["natura2000", "gold", "envo", "ncbitaxon"])
+    verdy.save(matches, "my_input.matches.csv")        # or .tsv / .sssom.tsv / .json / .xlsx
+    verdy.match.unresolved                             # ENA IDs ENA did not know
+    verdy.unmatched("my_input.csv", matches, ["envo"]) # names an ontology found nothing for
+
+    verdy.ONTOLOGIES                       # the ready ones; any OLS4 ontology ID works too
+    verdy.search("air temperature", "cf")  # one name in one ontology, its best terms
+    verdy.resolve("SAMD00132352")          # what ENA says about an ID
+
+From the shell:  verdy my_input.csv -o natura2000,gold,envo --out my_input.matches.csv
+"""
+
+from verdy.match import MATCH_TYPES, match, read_terms, resolve_terms, unmatched
+from verdy.resolve import SAMPLE_FIELDS, resolve
+from verdy.output import save, to_dataframe, to_sssom, to_table
+from verdy.sources import ONTOLOGIES, ontology_info, search
+
+__version__ = "0.1.0"
+
+predict = match          # the same: IDs or names in, predicted matches out
+
+__all__ = ["match", "predict", "resolve", "resolve_terms", "SAMPLE_FIELDS", "search", "read_terms", "unmatched", "save", "to_table", "to_sssom", "to_dataframe",
+           "ONTOLOGIES", "MATCH_TYPES", "ontology_info", "__version__"]

verdy-0.1.0/verdy/__main__.py

@@ -0,0 +1,60 @@
+"""Command line:  verdy TERMS -o envo,pato,cf [--column NAME] [--out FILE]
+
+    verdy my_terms.csv -o envo,ncbitaxon,gbif --out my_terms.matches.csv
+    verdy my_terms.csv -o pato,cf,uo                       # print a table instead
+    verdy --list                                           # the ready ontologies
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from verdy.match import match, read_terms, resolve_terms, unmatched
+from verdy.output import save, to_table
+from verdy.sources import ONTOLOGIES
+
+
+def main(argv=None) -> int:
+    ap = argparse.ArgumentParser(prog="verdy", description="Match your terms to the ontologies you choose.")
+    ap.add_argument("terms", nargs="?", help="CSV / TSV / Excel with an ID or name column, or a text file with one per line")
+    ap.add_argument("-o", "--ontologies", help="comma-separated IDs: see --list; any OLS4 ontology ID works too")
+    ap.add_argument("--column", help="the column holding the terms (found by its header otherwise)")
+    ap.add_argument("--out", type=Path, help="write here: .csv, .tsv, .sssom.tsv, .json or .xlsx (default: print)")
+    ap.add_argument("--per-ontology", type=int, default=3, help="at most this many matches per term and ontology (3)")
+    ap.add_argument("--min-score", type=float, default=0.34, help="least word overlap for a related match (0.34)")
+    ap.add_argument("--list", action="store_true", help="list the ready ontologies")
+    args = ap.parse_args(argv)
+    if args.list:
+        for k, v in ONTOLOGIES.items():
+            print(f"{k:11} {v}")
+        return 0
+    if not args.terms or not args.ontologies:
+        ap.error("give a TERMS file and --ontologies (or --list)")
+    try:
+        terms = read_terms(args.terms, args.column)
+        matches = match(terms, args.ontologies, per_ontology=args.per_ontology,
+                        min_score=args.min_score,
+                        progress=lambda d, t: print(f"\r{d} of {t} lookups", end="", file=sys.stderr))
+    except ValueError as exc:
+        ap.error(str(exc))
+    print(file=sys.stderr)
+    unresolved = match.unresolved
+    if unresolved:
+        print(f"nothing found for {len(unresolved)} IDs: {', '.join(unresolved[:20])}"
+              + (" ..." if len(unresolved) > 20 else ""), file=sys.stderr)
+    if args.out:
+        save(matches, args.out)
+        print(f"{len(matches)} matches for {len({m['term_id'] for m in matches})} of {len(terms)} inputs -> {args.out}",
+              file=sys.stderr)
+    else:
+        sys.stdout.write(to_table(matches, "\t"))
+    for o, missing in unmatched(resolve_terms(terms)[0], matches, args.ontologies).items():
+        if missing:
+            print(f"nothing in {o} for: {', '.join(missing)}", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())